Times when you wanted to cry

[iaind]

It looks like there's a lot of people in similar jobs to myself here, so thought it would be amusing to share stories of times you've found mistakes, misconfigurations and just general idiocy on your network.

I'll start - I've been doing some reconfiguration and rebuilding of our AD, File and Exchange servers and all was going smoothly until I finished the work on the file server. As it was previously a combined DC/File server and I moved to a dedicated DC and standalone file server, I gave the old servers IP to the new DC and set up a DNS alias for the new file server. All was working fine until I noticed one of our Citrix servers wasnt loading the roaming profiles and I was getting DSAccess errors on the old Exchange server. After a bit of probing, flushng DNS caches and rebooting, I finally discovered what had been done..

Someone had thought it would be a good idea to fill the hosts file on each server with entries for EVERY other server. An easy fix but I did want to cry, I dont understand how someone can be so stupid.

Can anyone beat that?

 

[iaind]

This is generally where strict change control and documentation comes into play, could have saved you a lot of time and effort.

I completely agree, although this problem dates back (the modified dates on the files were all 2006). Its something I've been pushing for but we're a small team (4 of us) so there's a bit of resistance to it. I cant see any point moving towards a full ITIL change control procedure, but a bit of control and documentation would help!

Oddly, the network was built as a 2003 domain, there's been no upgrades or migrations...

 

[iaind]

You have to wonder how some of these companies get away with it. A lot of these stupid mistakes seem to be done by consultancy companies who charget a lot of money for their services.
Another one I saw was an Exchange 2003 server that had been done by a fairly large, well known consultancy/support company. It was a 6 disk system and there were 3 mirror arrays configured labelled "system", "database" and "transaction logs". Fair enough, its an OK configuration I thought. On closer inspection, though, the "system" volume contained the OS and transaction logs, the "database" volume contained the EDB files and the transaction logs volume had the STM files! Thats a mistake thats so fundamental I had to double check I wasnt imagining it..

 

[iaind]

Fairly common

Yup, ive seen it a few times too. Often its some app that needs certain permissions to run properly, so instead of figuring out what permissions are necessary, they just set the user as a local admin..

 

[iaind]

Indeed; it's the default behaviour of SBS for a start.

If you "assign" a user to the computer, which I never do since I realised what it was going. Seems daft that a product designed to be easy and secure would do that!

 

[iaind]

Local admin rights to users is very common in my experience. So many applications/browser plugins etc fall over without it.

I'm not so sure, most of the problems can be removed by applying specific NTFS permissions or assigning the correct user rights assignment in group policy - a lot of admins are just too lazy to figure out what is required