Xeon Server Watercooling

If you have any kind of problem, who is responsible? Is it you? Servers are built by experts with 24/7 operation in mind and if this kit is in a small room, it doesn't matter whether the initial cooling is via water, you still have to cool the water. That room is going to get hot and once it does, the air cooling the radiator will be hot and the server will therefore be hot. It will probably still work, but if this server is a Production system, the argument for more money needs to be based on the business cost when it's not working, not when it is. I don't imagine this dual-quad-beast is cheap so what's the problem with some sound insulation and air-conditioning?
 
Yes i have considered how they will be drained, and yes it will be in a parallel loop but it will be done in such a way that they both get an equal flow, so that 1 processor isnt better cooled than the other.
 
MikeTimbers said:
If you have any kind of problem, who is responsible? Is it you? Servers are built by experts with 24/7 operation in mind and if this kit is in a small room, it doesn't matter whether the initial cooling is via water, you still have to cool the water. That room is going to get hot and once it does, the air cooling the radiator will be hot and the server will therefore be hot. It will probably still work, but if this server is a Production system, the argument for more money needs to be based on the business cost when it's not working, not when it is. I don't imagine this dual-quad-beast is cheap so what's the problem with some sound insulation and air-conditioning?
Click to expand...

I have to agree again.
 
its not a production server, this is only a small business, but the server is being built for to run some virtual machines so people can work offsite to input data to get their website up and running, once thats been done, and in the future things will be gradually upgraded and improoved, an air condioning system will probably be considered by the owners, a rack will be introduced etc, this is just to solve the problem of noise. no the server is not cheap, they are slowly upgrading and implementing everything to spread cost. Because i do agree air conditioning is required at some point, but as a small business its not mission critical.
 
RB2004 said:
Yes i have considered how they will be drained, and yes it will be in a parallel loop but it will be done in such a way that they both get an equal flow, so that 1 processor isnt better cooled than the other.
Click to expand...

I'm not saying that it couldn't be done, just that the simpler and far better long term solution is to put proper air-conditioning in the server room.

If this was smart, we would all be doing it, and I promise you, we're not. I've done dozens of server installs in the last 3 years and I always specify a small, secure room be made available, with redundant power and and externally vented air conditioning suitable to cool the room to 18C. I got the 18C from thin air, but it seems to work fine.
 
WJA96 said:
Seriously - have you really considered how you are going to drain this thing, or even properly leak-test it?
Click to expand...

Drain it using some Quick Relase Compression Fittings and leak test in the normal manner... whats so difficult there?
I've already told him on another forum to use compression fittings instead of bigger bore tubing like 1/2" as it's considered much safer.

I'll happily link you to a watercooled server but i'd get shot for doing so... although tbh it isn't the greatest of exmples and i'd have done it a bit different.
 
MikeTimbers said:
FWIW, servers (if from a name brand) generally have very efficient cooling systems and are specifically designed to handle high heat loads. Of course, they also tend to be seriously noisy, but any amendments of the sort you intend to make will invalidate any warranty.

1) Get a rack.

2) Get an air-conditioned server room.

3) Who cares how much noise and heat it makes!

I accept that items 1 & 2 might be more than a water-cooling solution ;)
Click to expand...


+1

You don't water cool servers!
 
WJA96 said:
Really? So the radiator doesn't actually drop the temperature as the water passes through it? I always thought it did so that the water was cooler when it left the radiator than when it went in?

How does it work then?
Click to expand...
The difference in temperature of water going in and coming out of a radiator is generally less than 1C, remember it's why loop order doesn't make any real difference.
 
I do this stuff for a living, and that makes me conservative. I'll never get sacked for buying Dell, rather than cheaper home-brew kit, even though I could build and maintain the home-brew as well or better than Dell look after their stuff, but I choose not to simply because of the difference in the amount if aggravation involved - one gets me blamed and involved directly, the other involves me calling Ireland and a man comes within 4 hours.

This looks like a good idea now, but give it a couple of months and you'll pay for air-con yourself.
 
WJA96 said:
I'm not saying that it couldn't be done, just that the simpler and far better long term solution is to put proper air-conditioning in the server room.

If this was smart, we would all be doing it, and I promise you, we're not. I've done dozens of server installs in the last 3 years and I always specify a small, secure room be made available, with redundant power and and externally vented air conditioning suitable to cool the room to 18C. I got the 18C from thin air, but it seems to work fine.
Click to expand...

I do agree that is a simple long term solution, and at some point air conditioning will be required, if its a large building or medium to large sized business then i would insist on a secure air conditioned room, insulated against sound etc. but because this is only a small business, they dont have the kind of funds for everything in 1 go, the server for the virtual machines is costing them a lot already, just so they can recuit enough people to get their website data input and launched, their product line is around 20,000 products although only a fraction of that is held in stock. So to buy racks, air conditioning, securing and insulating a room costs a lot of money which is why they are implementing things slowly and as required. And later on the room will be air conditioned, UPS will be installed, and room properly secured, but this is only a small premises and business.

If it was a secure room with soundproofing and airconditioning i would go for the air cooled solution, but because there isnt as of yet, and noise is an issue, as the room is next to their office they dont want the constant hum of 3 servers running, and because this new server will product a lot of heat water cooling just seemed like the ideal solution right now until things are properly sorted.

Once leak tested properly, and in my case i will even be air pressurizing it to test it first there is no reason why it should not leak, especially with the compression fittings as recomended, and it should just require maintainance to check cooland level, pump is running.

It is not mission critical.
 
The 2 links below, show there is a small market for water cooled systems, despite the risks, when they work, they do offer advantages, if a radiator and liquid wasnt better, why arent cars still air cooled? what if the hose leaks on the motorway and your engine overheats? why not just close the engine bay and fully air condition it?

The only reason why its slow on the uptake is because air cooling still works, in the correct environment, airconditioned etc, its failsafe, minimal risk, and a proven technology. and for most datacentres its easier to replace a server like for like, rather than rip out every server and change to watercooling. Also water cooled servers are high maintenance, pump can fail, coolant can need topping up. so Air cooling is more convienient.

But when you put an watercooled server in an air conditioned room the servers will run cooler than air will ever cool them. And whatever you argue, companies are starting to develop water cooled systems, IBM and HP are developing water cooled racks. Because, if they can be made to work they do offer advantages otherwise they wouldnt bother. and if they dont leak to begin with chances are they wont in future unless disturbed, and distilled water is not electrically conductive nor are the specialized synthetic coolants.

New Water Cooled Server with Intel CPU - Nec innovates on the server market

http://news.softpedia.com/news/New-Water-Cooled-Server-with-Intel-CPU-12866.shtml

Boston unveils water-cooled server

http://www.pcpro.co.uk/news/142818/boston-unveils-watercooled-server.html

I quote
"Our tests show that SuperServer 6015TW-TS reduces power consumption by over 30% compared to rival server solutions and doubles the rack installation density by removing much of the heat dissipation issues arising from dense computing installations", claims Manoj Nayee, Director of Boston Limited."

Also the interior of the case stays cooler, because hot air is not circulated around the case from the processors, it is absorbed by the water, and disapated straight outside the case by external rad, or pushed directly out when rad is mounted on 1 side of the case.

I didnt create a thread to argue, i have looked into the pros and cons, i reliase what an "ideal" and perfect setup is, and completely agree with what people are saying, and in time i will push for the air conditioning, insulation etc, but for a small business it has to be done bit by bit.

All i wanted was recomendations, and advice on the setup, good products, bad products, things to look out for or do, and i have taken note of all your sugesions and thank you for them.
 
Last edited:
No-one's looking to upset anyone. This is an interesting debate.

What you're proposing is interesting but as you can see, others of us who look after servers are doubting the wisdom - not the technology - of this. I'm guessing you don't actually work for this company but are some sort of 3rd party contractor? If so, I can only hope you're going to charge them a suitable amount for this peace and quiet you're going to give them.

I didn't know NEC did water-cooled servers! Good for them. The second link is interesting in that although it's a small supplier, they've done it in a 1U chassis albeit with external radiators. Take a pointer from that one. Quick release connectors so that the chassis can be disconnected and removed easily. Have you considered getting another rack-mount chassis and putting the pump/rad in there? It wouldn't add much to the overall cost.

I still wouldn't do it though.

If I had to, I'd just ensure the case I used had big fans so I could run quiet ones and rely on really good case cooling to keep the temperatures below throttling level but not by much. I've seen servers run at >80C quite happily for years.
 
Last edited:
The difference I can see, correct me if I'm wrong, between those articles you linked and your planned DIY solution; is an off-the-shelf water-cooled server will still have its warranty.

Now, I don't know this for certain, but, I'm guessing Dell (or any supplier/manufacturer like them) would kindly tell you to go away if they found a fault was caused by you chopping their product in to pieces.
 
Did you get anywhere with this? I'm building a system for 3D reconstruction which has such requirements for RAM and CPU/Graphics power that only a few server boards are up to the task. I have 771 socket E4000 series CPUs, but I can't figure how the D-Tek FuZion V2 blocks will mount to the motherboard.
 
As mentioned by others, I do find the idea of water cooling serves a bit worrying as it almost certainly complicates upgrades and replacements. Also as already mentioned it does solve the problem of heat dissipation from the case, but not the larger problem of the heat build up in the room its situated in.

The other major caveat in my opinion is that if this is designed to keep this machine quiet a lot of the servers we have at my opffice actually produce more noise from the triple redundant power supplies than the whole of the rest of the system put together. Therefore unless you intend to water cool the PSUs I'm not sure you will solve the problem.
 
It's not a server, it's not overclocked. I've built several of these machines for various research groups, but they all complain that they are too noisy. There's no alternative to having the machine in the room they are used in, because of the high-end graphics cards - they usually drive 2 of 30" TFT displays. I am trying to build this system to be more liveable with! There's no issue with heat build-up in the offices as they are large, open plan rooms with air conditioning.
 
grantwray said:
It's not a server, it's not overclocked. I've built several of these machines for various research groups, but they all complain that they are too noisy. There's no alternative to having the machine in the room they are used in, because of the high-end graphics cards - they usually drive 2 of 30" TFT displays. I am trying to build this system to be more liveable with! There's no issue with heat build-up in the offices as they are large, open plan rooms with air conditioning.
Click to expand...

You can use long DVI Leads (maybe with DVI-DAs to extend cable runs if required) or even DVI over CAT5 to move the machines away from the operator. We have several very powerful machines for video edit suites all multiprocessor multi core boxes with very silly large and powerful external RAID arrays. All very hot and noisy!

They live in an air conditioned machine room!

The operator sits in a nice quiet room elsewhere in the building, comfortable, quiet and without the slightest sign of the machine other than monitors, mouse and Keyboard.

I do realise its not always possible, but if can be done its the best route.

E-I
 
You must log in or register to reply here.
Back
Top Bottom