Windows


A friend came to me this week with an issue he had earlier – he had to change the subnet of a client’s network, and issues with the Windows 2003 Domain Controller appeared. Here’s how to avoid (or fix) the problem. For this HowTo, I’ll be using a current network of 192.168.1.2/24, with the server being it’s own DNS and WINS, and a gateway of 192.168.1.1. We’ll be changing the server to 172.31.2.2/24, and using the domain name “foobar.local”.

  1. Why are you changing the subnet? Most often, I’ve had to do this for customers because a business partner’s subnet has conflicts that are coming up during VPN tunnel creation. I’ve had other reasons, and you might too – but that’s the most common I’ve seen – 2 sites that are both 192.168.1.0/24 trying to build a VPN tunnel to each other.
  2. Now that you have a good reason to go through the pain, determine your layout – I’m going to write this for a single DC environment and point out a few changes for a double-DC environment. If you have more, you should be able to extrapolate the requirements from there, but you can also leave some comments and I can write another post if required. I’ll also be writing cmd scripts for most changes, rather than attaching huge (pixel-size) images.
  3. Determine the new subnet.
  4. Add the reverse lookup zone to DNS for your new network. In this case, in the DNS wizard, you’d make the reverse zone “172.31.2″ (filling in all boxes). This makes the zone name “2.31.172.in-addr.arpa” or “172.31.2.x” in the DNS console.
  5. Open Active Directory Sites and Services. Right-Click “Subnets” and add in the new subnet (172.31.2.0 with a Subnet Mask of 255.255.255.0). You’ll see the subnet listed as CIDR notation in the box (172.31.2.0/24) for verification. Pick the site that the new subnet belongs in (probably Default-First-Site-Name), and click “OK”. This guarnantees that AD will recognize the new site properly.
  6. Determine the new IP for the DC, and *add* it to the DC’s adapter. Do NOT remove the existing IP, yet. Don’t add the new gateway, yet. Just add the new IP, and save the settings.
    netsh interface ip add address "local area connection" 172.31.2.2 255.255.255.0
    (This command assumes that your DC has a static IP address. I haven’t seen a site use DHCP for their DCs yet, but it’s a possibility. This command on a DHCP address will set only a SINGLE address with no gateway, and could leave you with 0 remote access to the server.)
  7. Verify that the new IP is showing up on the server.
    ipconfig
  8. Get the server to register its new info in DNS
    ipconfig /registerdns
  9. Wait and watch the Application and System event logs for DNS related errors. Also check that the DNS server is publishing the new IP address, not just as the server name, but also as the domain name, and the DCs (This can take up to, but shouldn’t take longer than, 15 minutes):
    nslookup foobar.local
    Also, open the DNS console and look inside “_msdcs.foobar.local” to see that the GUID of the server is listed with both IPs.
  10. Once the server is advertising the new IP, you can swap the system to use the new IP range completely. It’s not time to remove the old IP yet though. This is the time to change the gateway, DNS servers, and WINS server. Since the server is listening on, and advertising on, the old and new IPs, DNS shouldn’t have any issues.
    netsh interface ip set address "local area connection" static 172.31.2.2 255.255.255.0 172.31.2.1 10
    netsh interface ip add address "local area connection" 192.168.1.2 255.255.255.0
    netsh interface ip set dns "local area connection" static 172.31.2.2
    netsh interface ip set wins "local area connection" static 172.31.2.2

    This can, like all other steps, also be done in the GUI quite easily, by just shifting the orders of some things (IPs), and replacing others (DNS/WINS/Gateway).
  11. Now re-register the server with itself, looking for error messages in the Application and System logs.
    ipconfig /registerdns
  12. Remember that changes to DNS can take up to 15 minutes to appear, as you watch the logs for errors, and check dns.
    nslookup foobar.local
    What you’re looking for in the nslookup is to see the address of every domain controller in your domain. If this is a single server (like a Small Business Server 2003 network), you should, at this point, see 2 or 3 addresses (depending on how you set up your public network, for SBS2003).
  13. If everything looks good here, this is a good time to test some logons to make sure things are working properly. This is the point in the project where I normally create the new DHCP scope, deactivate (not delete) the old scope, and change the LAN settings on the router. This is also a good time to reboot some of the client PCs to make sure that they can boot up properly, get IP addresses in the new subnet, login without errors (remember to check that Application log!), and get online without issues.
    Because we haven’t removed the old IP from the server yet, the biggest issue you *should* run into is a client who gets an IP from the old subnet, or is statically set. They’ll log in ok, but won’t be able to get to the internet (unless you’ve got SBS2003, and the server is also your router). At this point in time, it’ll be easy to figure out if the new IP is working – clients that can log in and get access to resources are logging in to the IP address for the DC that’s local to them – if they’re in the new subnet, then your new IP is working. You can now reboot your DC as a final test (or act of faith, as your experience may prompt).
  14. When the DC comes back up, log in, and remove the old IP address – this is easiest in the GUI, but if you’re doing it with netsh, I prefer to just reset the DC settings completely.
    netsh interface ip set address "local area connection" static 172.31.2.2 255.255.255.0 172.31.2.2 10
    ipconfig /registerdns
    nslookup foobar.local

    Now it’s cleanup time.
  15. Open Active Directory Sites and Services. Delete the old subnet.
  16. Open up your DNS server and make sure your forwarders are correct – lots of small offices skip this step – your forwarders should be the IP addresses of the DNS servers that your ISP gave you. Don’t put these into your DHCP scope DNS servers list, or even in the list of DNS servers on your server – they go in the “forwarders” section here.
  17. That’s it. Your clients are set up and ready to go, your server is healthy, and now you get to tackle the problem that made you have to change the IP in the first place!

For those who are having issues, because they’re finding this after attempting the change, or for my friend who prompted me to write this, here are some suggestions:

  1. Add the old IP back to the server, run <code>ipconfig /registerdns</code> and wait for the old IP to take. Now verify that the server’s logging in properly, and not giving error messages.
  2. Do you have all the zones in DNS, so that registration can take? Many small sites forget to put in the reverse zones into the DNS server.
  3. Go back and check Active Directory Sites and Services to verify that both the old and new subnets are listed. While you’re in transition, all subnets should be listed in all locations.
  4. If your server is screwed up badly enough that you can’t even log in, boot into Active Directory Services Restore Mode, make sure that your networking changes are set properly, and that your DNS server has it’s forwarders set up right. This is a good way to check the base level of your server’s health, then bring it back online to try to log into AD.
  5. There’s no reason to need to rebuild the server, but if it’s a small enough location (AD doesn’t change very often, or if it does, the changes are minor), and you have good backups, take a look at how long this is taking you, and how much longer you’ll spend attempting to fix the problem. Going back to the backup from the previous night and restoring ONLY the system state in Directory Services Restore Mode might be the fastest and best solution. Then you can follow the steps in here in order, and grow fewer ulcers.

Next week sometime I’ll do a similar version for multi-DC, multi-Site AD networks – it’s a lot shorter and easier.

We had to emergency move some SQL Server service account mailboxes to a new Exchange server tonight, and dump all the mail in the mailboxes. Since doing that, some of the servers have been randomly failing SOME of their jobs – it looked exactly like the issue described in FIX: SQL Mail Procedures May Intermittently Fail with Error 0×8004010f , but we’re running SQL 2000 SP4.

We fixed it by having the DBA run:
xp_startmail
xp_stopmail

Then re-running the jobs. It seems to have worked, but I’m curious how we got hit by a fixed bug. My only thought is that it must have something to do with SQL thinking we essentially truncated the mailbox.

On anonther note, I’m writing up full instructions for how to change the IP on a DC, particularly for small offices with only 1 or 2 DCs.  It’ll post by end of the weekend.

On Friday we successfully executed a 20-server move from one colocation facility to another across town.  The new site has much more room for expansion, is more secure, and will save the company a few hundred thousand dollars a year in related costs.  So it was a *good thing*.

Technically, the best part about it was being able to perform the move during business hours.  There is enough redundancy in the systems now to allow the full shutdown of DCs, an Exchange server, a huge portion of the Cisco Call Center environment, other support and security systems, and the end users never notice.  That is a fantastic feeling, knowing that designs are coming together properly to allow such controlled failures, without affecting the business.

Over the course of this move, I’ve learned a few things:

  • Exchange servers appropriately update themselves upon an IP subnet change with no errors.  I expected a 2nd reboot to be required after the server recognized it was in an completely different AD site.
  • Physically moving DCs to a new site is also extremely easy, provided you update DNS / WINS appropriately.
  • Cleaning up bad subnets in AD Sites and Services is a pain, because it’s so tiresome doing all the subnet calculations over and over again.  Worth it, but boring.
  • Security guards at building docks can be real jerks, or can be really easy to work with.  Leaving the building, the movers kept having to drive around the building as we went back in for another pallet’s worth of equipment.

Now it’s time to rebuild servers that have been brought back to the office to be re-deployed with new OS’s in the DMZ, and finish building that environment properly!

I have a Sprint PCS PPC-6700 Windows Mobile 5 PDA/Phone. The battery life on it is decent, considering all that it does, but I still dislike it.

Here’s why: it’s a terrible consumer device. It’s great for geeks like me who understand software is buggy and difficult to write, but I couldn’t imagine any of my end users dealing with this thing. Examples:

The phone is ok, but the Bluetooth radio randomly shuts off on me. The worst is in the middle of a conversation, my headset will go silent. And it’s ONLY ever in the car. So there you are, driving down the street, and having to fumble for a soft-button to turn on the speakerphone. Many MANY people will mention something or other about software upgrades required, and believe me, I have them. I had them before Sprint support knew what I was talking about. I’m still 100% up to date, according to Starcom’s site. No, I’m not going to run a 3rd party update to a business tool, like I would my own personal phone.

The device requires a reset every 3 days. I have 2 pieces of add-on software, and I’ve actually REMOVED 8 installs from Sprint (from the hidden ROM, so they don’t come back during a hard-reset). I installed Google Maps, because it’s fantastically worthwhile, and “TCPMP” for media playing (Windows Media Player doesn’t play Cisco UNITY voicemails sent to email). Out of the box, it required a reboot every 1-2 days. After 3 days of no reboots, ExchangeActiveSync stops reliably retrieving email. The touchscreen becomes wildly out of sync (I can’t hit the scrollbar anymore, usually). Phone calls actually hang up in the middle of conversations. And the phone finally won’t respond to any buttons or screentaps, save for the “power” button.

It Phantom-Dials. I’ll read an email, put the phone back in its holder, and walk down the hall. 45 seconds later, one of the recipients of the email will call me back asking me what I called them for. Best I can figure, is that the joystick is getting depressed, scrolling up, getting depressed again (on a recipient of the email), and the call button is getting hit. In the holder designed for the device, on my hip. I’ve been wearing cell phones on my hip for work since I was 19 – don’t tell me I’m hitting it with my elbow. Having to lock your phone in your pocket is understandable. But in it’s designed holster?

So, what DO I like about the phone?

I don’t have to open my laptop at home to watch email every night, in case something breaks. Exchange ActiveSync DirectPush in conjunction with SMS for alerts is fantastic. I turn on the sounds for text messages, but not email, and if it’s an actual issue, I get the alert, hear it, and respond. But if it’s a co-worker sending out a “I finished doing this overnight work.” email, I don’t get woken up. Better response time for the business, more sleep for me.

Charges from USB on my laptop. On the road, this has been a life-saver.

Can install all sorts of neat software. Games, document readers, etc. Adobe Acrobat for PPC and Microsoft Reader, combined, give me hours of reading material. Especially manuals for things I have to do the next day.

Pocket IE: www.weather.com, www.opentable.com, www.google.com (which has a great mobile device reformatter). I took just my phone to New York City for a 3 day weekend, and was able to book every dinner, find directions, and verify that Ferries were open on the days I wanted to go places.

Google Maps. I use this to check the traffic of my drive on my way to the car in the parking lot, so I can decide which way to drive home. Add accurate directions that I can pull from my contact list… wow.

Now I just have to figure out RAPIP, so I can plug it into my Ubuntu Feisty laptop and sync THROUGH the GNU/Linux OS, rather than just through WinXP.

« Previous Page