Show Us Your Racks

[Garp]

When I can pay enough people that I never have to venture in there, I'll be a very happy nerd .

I'm a happy nerd to have reached a skill level where I no longer have to work in the DC, and only ever venture near it once in a blue moon (since changing departments into sysadmin I've spent maybe 3 working days tops in there, and 1 days worth of time have been when I've been stuff on my own personal server that I couldn't do remotely) That's gone from working there 2 weeks in 3. What a godsend it is

 

[jabroni_83]

just wondering - what are people using to monitor the health of their hp and dell servers?

 

[Worthy]

We went for HP because when I call them with a problem, I have an engineer on site within 6 hours, Dell only seem interested in getting you to run diagnostic tools for days on end. I don't have time for that and HP get things fixed quickly, it was an easy decision in the end.

I'm not sure what support you paid for, but Dell have a 4 hour response time.

 

[bigredshark]

I'm not sure what support you paid for, but Dell have a 4 hour response time.

When they feel like it, it's not like we're small, we have hundreds of servers deployed and whenever I've had to deal with Dell they've been hopeless, asked us to run diagnostics, provide logs. When I call HP and tell them the motherboard is dead, they say they'll have the part despatched and an engineer on site straight away. No logs, no 'run the diagnostic tool and email us the results', fixed, now! I don't have time to mess about so HP (or Sun on occassion) is all I'd touch these days - maybe Dell have changed but they disappointed me once and they've lost my business.

 

[Baz]

When they feel like it, it's not like we're small, we have hundreds of servers deployed and whenever I've had to deal with Dell they've been hopeless, asked us to run diagnostics, provide logs. When I call HP and tell them the motherboard is dead, they say they'll have the part despatched and an engineer on site straight away. No logs, no 'run the diagnostic tool and email us the results', fixed, now! I don't have time to mess about so HP (or Sun on occassion) is all I'd touch these days - maybe Dell have changed but they disappointed me once and they've lost my business.

I am with you, once I bought four Dell laptops, about 5 years ago, one went wrong after 11 months and they didn't want to know, if they could only just be bothered to fix the laptop (eventually) then I didn't want them as a server provider.
HP are brilliant, I know computers will go wrong, but HP's customer service is great, I have sent a few workstations and notebooks back and all replaced or fixed very quickly, and when I have had a server issue, they sorted the problem out the next day (didn't have 4 hour response on that one )

HP all the way here.

 

[Schnippzle]

I work for the NHS in IT (only helpdesk/techie stuff) and have to go to our servers rooms now again. They are horrible places. I really feel for the techies who have to work in there for any length of time.

That said though, there is some pretty neat kit. We've a SAN with god-knows how many HDDs in it. Mostly Dell and IBM stuff.

The best thing about the server room is a pc that isn't Websense'd. Full internets at work!

Im also in IT for the NHS but for the Liverpool PCT. Server room here is pretty serious but Im afraid I cant post any pics of it.

 

[NickK]

Tis a pity I've not got photos of the last capacity expansion for a customer I was invovled with - 27 racks with varied gear including servers, SANs and a load of cisco kit. Actually so much HP gear that they had to reserve production capacity

 

[Confused]

Here's the one we've got in the main comms room @ work.



Blue = general data
Orange = servers
Yellow = voice
Green = IPTV
Red = uplink, leased line, important stuff etc.

I've been the only one who's put any cabling into this cab, hence the neatness! The other cab at this site is a little worse for wear - it was a rat's nest before I started, and it hasn't been stripped and redone since!

 

[BoomAM]

I find Dell quite good for workstations/laptops.
I do have to spend 10-20min going through the problem with them, but usually they send out an engineer with replacement parts the next day.

Having said that though, their 'next day' has turned into 'the day after next day' as of late.

Cant comment on their server stuff though.

 

[Garp]

HP can be a real pain in the arse despite how much money we spend with them each year (we're starting to kick up a fuss and getting noticed, though.) Particularly can be a hassle dealing with their Indian call centre staff, though to be fair I think I'd probably have the same hassles with UK based staff.
To whit:

Me: We've got a server on CarePack blah blah blah that is reporting a faulty fan

HP: Hmm. I see that's an xxxxxx server, am I correct?

Me: Yes, that's right.

HP: Do you have the part number for the fan?

Me: No, its a live system with 2 CPUs, one of which has been disabled since last reboot due to the fan failure I presume. It's not a server I'm in a position to be able to shut down.

HP: I'm sorry sir, I can't help you unless you provide me with the part number for the fan.

Me: It's a bog standard xxxx server with a P4 blah blah processor in it. Serial number xxxxxxxx, surely you can tell from your records what CPU Fan it should have on it?

HP: I'm sorry sir, I can't help you unless you provide me with the part number for the fan.

Me: Its not a server we can shut down, except for doing the fan replacement. Are you seriously telling me your fans are non-standardised components

HP: I'm sorry sir, I can't help you unless you provide me with the part number for the fan.

Anyone spot the script monkey? Dell on the other hand, I can get part numbers and details down to minute detail for every server we have just from the code on them.
Their support can be extremely dodgy, and IME their hardware just doesn't last under load anywhere near as long as HP stuff does. Incidentally that above incident was a bit of a one off, generally HP support is superb. That particular fault took 5 engineers and 6 replacement parts to finally fix.. eventually the final senior field engineer arrived with almost a complete replacement server, and he sussed the others were all muppets and had been replacing the wrong fans entirely.

The other big hassle I've had with HP in the past is explaining the simple concept of "Dual Channel Memory". HP assign component codes to all compatible memory by size and type, e.g. every PC-3200 1Gig chip has the same product code, regardless of manufacturer. They appear to go with the "whichever named brand is cheapest" approach to memory too. The pairings you'll get with each server will be identical, but God help you if one of those chips fails. I've had them sending Crucial 6-4-4 CAS3 chips to be paired with existing Samsung 6-2-2 CAS2 chips, and then them being baffled why the motherboard has decided not to run them in Dual Channel mode.

HP: I'm not sure why it isn't in Dual Channel mode sir

Me: I am, its because the memory chip isn't a compatible match

HP: But the product number is the same

Me: Yes, but the chip is rather different performance wise, enough to stop Dual Channel anyway

That's happened so many times it's become a matter of routine, all the engineers know to look out for that one and to warn the HP support desk about it. Even the HP engineer that we routinely end up with knows to bring multiple types of memory chips whenever he's dealing with memory faults.

 

[memyselfandi]

garp although you make it sound like an awfull job, personly i would still love to do a job like that. And you mention that it doesnt take a great deal of skill, but what about when you need to upgrade all your routers/switches to a newer model (i know this wont happen often, maybe once every few years, or sooner if you are constantly expanding) in a data centre environment i bet someone with just a CCNA wouldnt get on too well with replacing the old routers with new ones (unless of course its a simple matter of transfering the settings from old to new instead of creating the settings from scratch).

Hmmm ... DC staff at our place would not have much, if anything, to do with any of that sort of thing. Network related things are covered by Network Support and server related things by the appropriate Server Support team, both of whom can do most of their day to day work remotely, (I haven't needed to be in a DC for over 12 months). DC staff tend to be Operations who do things like the following; release batch jobs on mainframes, callout support teams if there are overnight issues, take third party engineers to the appropriate servers and load/unload tapes. It, apparently, is extremely boring. At our place, at least, you do not get involved in server or network upgrades.

OSA forbids me from posting photos or any specific details.

 

[Bakedgoods]

Depending on the size of where you work, network guys will often never have anything at all to do with servers.

I work at a fairly big site and like said above, there is a server support team for any server work, and a network team for anything relating to the network (switches, routers, infrastructure etc).

 

[Scottly]

Just out of curiosity, where do the rest of you who can't show your racks work?

Just think of anywhere sensitive data of national importance might be stored, and you'd probably be close enough.

I've worked in some ultra-secure places as a contractor, and let me tell you they are a nightmare. First off you have to get security clearance, which can take 6 months and requires background checks with the police and MI5, sometimes if you've travelled abroad they'll check with foreign intelligence services to make sure you're kosher.

Then you have the sites themselves, which usually have layered security. The first layer gets you access to the building. You have an ID check with the front desk before you're allowed in, but apart from that security at this level is about the same as working at a major bank - you have very limited internet access and no email access apart from the account they issue you with.

The second layer gets worse. You have your laptop and mobile phone taken off you and put in a locker, and are issued with a standard build laptop and phone to conform to their standards. Any software you wish to bring in is checked out by their security guys, which can take a while.

The final layer gets you access to the data center floor. Remote access to the servers is completely disabled, if you want to work on them you have to sit at a terminal at the end of the rack. You're not allowed to take anything in with you at all - you will be searched, have the laptop and phone taken off you, any documents will be confiscated. Anything you plan to do to a live server will have had to be planned to the exact keystroke, with backout plans, and will have to go through approval before you do it - which can potentially take weeks. Any software you want on the shop floor goes through an ultra-deep virus and malware checking process, which can take a week. Finally, they operate an airlock system. You are put into an airlock and weighed before you enter, and weighed again as you come out. If there's too much of a difference? Strip search, and full body cavity search if they think there's a need.

To get from the car park to the shop floor takes a bare minimum of 2 hours, and maybe an hour to get out again. The lengths they go to to protect their environment are extreme - doing something as simple as diagnosing and fixing a misconfigured IP address can take 18 months.

Yeah, these are awful places to work. But the pay makes up for it.

 

[fonzee]

Just think of anywhere sensitive data of national importance might be stored, and you'd probably be close enough.

I've worked in some ultra-secure places as a contractor, and let me tell you they are a nightmare. First off you have to get security clearance, which can take 6 months and requires background checks with the police and MI5, sometimes if you've travelled abroad they'll check with foreign intelligence services to make sure you're kosher.

Then you have the sites themselves, which usually have layered security. The first layer gets you access to the building. You have an ID check with the front desk before you're allowed in, but apart from that security at this level is about the same as working at a major bank - you have very limited internet access and no email access apart from the account they issue you with.

The second layer gets worse. You have your laptop and mobile phone taken off you and put in a locker, and are issued with a standard build laptop and phone to conform to their standards. Any software you wish to bring in is checked out by their security guys, which can take a while.

The final layer gets you access to the data center floor. Remote access to the servers is completely disabled, if you want to work on them you have to sit at a terminal at the end of the rack. You're not allowed to take anything in with you at all - you will be searched, have the laptop and phone taken off you, any documents will be confiscated. Anything you plan to do to a live server will have had to be planned to the exact keystroke, with backout plans, and will have to go through approval before you do it - which can potentially take weeks. Any software you want on the shop floor goes through an ultra-deep virus and malware checking process, which can take a week. Finally, they operate an airlock system. You are put into an airlock and weighed before you enter, and weighed again as you come out. If there's too much of a difference? Strip search, and full body cavity search if they think there's a need.

To get from the car park to the shop floor takes a bare minimum of 2 hours, and maybe an hour to get out again. The lengths they go to to protect their environment are extreme - doing something as simple as diagnosing and fixing a misconfigured IP address can take 18 months.

Yeah, these are awful places to work. But the pay makes up for it.

...yet financial details on 25 millions UK residents manages to leak out.

superb

 

[Scottly]

...yet financial details on 25 millions UK residents manages to leak out.

superb

This was stuff more important than your financial details.

 

[mrbios]

for you lot who work in the larger companies/data centers.....whos paid more? server support or network support? im going to guess it would be server support but would like to know

i currently do EVERYTHING to do with our servers here (albeit theres only 22 of them) and make sure the daily running and fault finding of our network goes to plan, any software that needs installing on client machines i have to do as well (thank god for ghost)

so naturally im curious as to how little im being paid compared to what someone in a similar position but only has to specilise in one of the roles gets

 

[A[L]C]

Network engineers tend to be better paid.

How much do u get?

 

[Bakedgoods]

I'm envious of the experience some people get at smaller places. They have to know a LOT more and have a wider knowledge base. It's really worth doing something like that for a couple of years, the experience you get would be very valuable.

When you work at a larger place it's much more specialsed. The network team don't have time to be worrying about looking after servers as well.

 

[memyselfandi]

for you lot who work in the larger companies/data centers.....whos paid more? server support or network support? im going to guess it would be server support but would like to know

i currently do EVERYTHING to do with our servers here (albeit theres only 22 of them) and make sure the daily running and fault finding of our network goes to plan, any software that needs installing on client machines i have to do as well (thank god for ghost)

so naturally im curious as to how little im being paid compared to what someone in a similar position but only has to specilise in one of the roles gets

Probably about the same on average ... I can't speak in detail for Networks but on the Server side there is quite a range from pitifully low to quite reasonable ... we all think we should be paid more though

 

[memyselfandi]

I'm envious of the experience some people get at smaller places. They have to know a LOT more and have a wider knowledge base. It's really worth doing something like that for a couple of years, the experience you get would be very valuable.

When you work at a larger place it's much more specialsed. The network team don't have time to be worrying about looking after servers as well.

hmmm ... I don't know about that, I am in a server support team looking after Unix kit but I have to be familiar with the Wintel boxes and network and SAN configuration otherwise it would be a nightmare troubleshooting problems which involve other areas, (which they are trying to blame on us )