watercooling a renderfarm

robinlawrie

robinlawrie

robinlawrie

Associate
Joined
26 Oct 2009
Posts
93
Location
London
Ok so ive been comissioned to put together a custom 10 node overclocked renderfarm, using microatx boards (gene2) and some nice oc'ed 920's, these will be arranged closely packed, with a right angle adapter for the pcie16 slot, and space for big ass graphics cards on each, for future upgrading (gpu rendering is aaaalmost on us....)

cooling wise, im gonna cool the cpus and northbridges initially, and consider the gpus later.

i was gonna go for one or 2 of those big alphacool ev01080s on either side of the case, venting air front to back thru the stacked motherboards.

however, he's now told me he wants it to be transparent perspex, and silent, and act as a pedestal for his glass meeting room table.


my current thought is to run some nice pipes up to the cieling from the middle of the table, across the cieling, and down to a seperate beefy cooler.


can anyone offer advice on the concept of using a central heating pump (quiet, powerful and long lived) and a nice unobtrusive house radiator to dissipate the heat?

im not sure how to do the calculations for flow rate and cooling power etc.

i had a quick look at homebase, and home radiators have a nominal power rating, usually around 1kw. does this mean i could dissipate 1kw of heat using a single home radiator?

(yeah i know, it will make his room toasty in summer.....)

i googled, but so far didnt see anyone using a setup like this, only car rads, which are a bit fugly for a meeting room.

any advice greatly appreciated!

cheers, Robin.
 
The 1080 radiators performed pretty badly under testing relative to thermochills, so you may want to give that part a miss.

The domestic radiator will indeed dissipate 1kW of heat. However the water will be at about 80 degrees while it does so, and computers really don't like 80 degree water. If you go for passive it'll need to be more than one domestic radiator. If you'd like a comparison, a thermochill triple radiator with 1600rpm fans will dissipate 2kW if you're willing to let the water inside it get up to 60 centigrade. Your best option for silent cooling is to have noisy cooling somewhere far away, with long pipes. Ideal would be a heat exchanger which attempts to warm up a swimming pool or perhaps a cold water tank, which it'll fail woefully at since worst case scenario is 5kW heat dump and that'll do nothing to a swimming pool.

He doesn't want perspex, it scratches far too easily.

A central heating pump will certainly do it, but be very careful here. I imagine they run at high enough pressure to cause tubing to leave the waterblocks it's supposed to be attached to.
 
interesting.. so the main problem (apart from crap from the radiator in your cooling blocks) is that the loop wont get hot enough for the radiators to be efficient. i wonder if this would still be the case with 10 oc'ed i7's ?

and other than a heat exchanger, is there any other way to avoid the crap buildup? any way to clean or treat the water / radiator so it stays clean?
 
yeah definitely not perspex for the tabletop, but as the unit its self will be central, under the table, scratching aint too much of a problem i think.

regarding the CHP, i guess if i split the flow 10 ways to the 10 nodes, the pressure going to one shouldnt be too high.

hmm, its tricky cos i have a feeling to really know, ill have to build it, but if it doesnt work..... ehmmm.

ok so maybe i ask him if he has an outdoor space behind the office.. i could hang a big ugly car rad out there. definitely limited options since its a central london office and im probably gonna have to assume he cant drill holes to the outside in a rented meeting room.
 
I'm fairly sure a dozen i7's will get the water hot enough for the radiator to be effective, but I don't think you'll like how hot the water is. Specifically overclocking won't like it. The man wants to cool over 3kW silently, got to compromise somewhere.

If you run them in parallel the pump won't need much pressure, but flowrate through each arm will be very poor (again, unless the pump is powerful). I wouldn't like to guess how much of a negative impact this would have on temperatures, an immersible pump in a tank of water feeding all ten in parallel probably makes the most sense. Some sort of manifold under the table to split the flow ten ways and a second to recombine it.

Could have a lot of radiators with slow rpm fans, blow a warm draft over the feet of people standing by the table?

Domestic radiators are probably aluminium, though they might be steel. Definitely not copper, so electrochemical corrosion could be a real concern. Still, anticorrosion additives are normally pretty colours.
 
Central heating radiator outputs are worked out using water at a set temperture, thats why heat outputs vary for a certain size because different companies use different temperature water.

If you're looking at using a heating rad for cooling id suggest you try and find an old, low water content one, you may still be able to get these new but iver never fitted one.

As for using a central heating pump the pressure will be a problem, id do some tests using a restrictor plate after the pump, you could use the pump valve for this.
Might also be wrth contacting Grundfos and seeing how they're pumps react to being undervolted.

Edit re. above post; rads are still made of steel.
 
thanks for the help... i think im getting the issue now.

so.. hmm yes there is space in the bottom of the machine for a stack of rads.. ive managed to fit the 10 nodes, plus full sized gpu's into a box 35 x 35 x 46, which should be fine as long as its well cooled. question remains which rads.. i guess the thermochill triples are your choice.. like to hazard a guess how many i can get away with for 10 nodes and relatively low fan speeds? (please dont say 10.. hehe.)
 
one other question, since big fans equals quieter (but lower static pressure iirc) i could stack 2 or 3 rads and have one huge fan behind them (300 mm or something) are there any sources of decent fans this size that run quiet and work ok?
 
hmm i was thinking more of 3 triple radiators side-by-side, with one 300 mm fan replacing the 9 120mm fans (with a shroud i expect) rather than sandwiching fans between rads..
 
I get what you mean now.

you're probably better with multiple little fans, slow rpm and quiet, one big one may concentrate the air on to a small area of the rads
 
Relatively high static pressure will be important to you otherwise the air just won't move through the radiators. I don't think I'd be happy with less than a 120mm rad per CPU, especially not if overclocking, so maybe four triple rads..? in context, I'm shortly to fit a triple 120mm rad for my CPU and northbridge, although granted I doubt you'll be generating quite as much heat per machine.
 
Do you have any pictures of the room this will be situated in? This might open up possibilities of something a little exotic such as under floor cooling or a passive skirting board rad.

I imagine polycarbonate would be better than perspex for the case if that were something else you'd be looking into.
 
What about the Koolance ERM-3K4U5 cools up to 20 CPUs apparently! Expensive stuff though at around £800. Dunno where you would get one though as Koolance state that you need to order 20 units or more! Worth giving them a ring though.
 
A 240mm radiator is the commonly accepted minimum for watercooling an i7, but the hundreds upon thousands of people using corsair H50s would suggest you can get away with a 120mm per processor if high end air temperatures are sufficient. The biggest thermochill radiators out there are the quad 120mm radiator and the triple 140mm. Some useful performance data on triple 120's here.

I wouldn't like to guess how a single 300mm fan would perform. For obvious reasons you'd need to seal the gaps between the radiators. I'm sure my 38mm thick 120mm fan blows more air than my desk fan, hard to guess which is louder. I would indeed suggest a layer of radiators either side of the fan. This isn't brilliant for performance, as one set of radiators gets feed warmer air from the first. However I believe it does make things quieter. Probably only useful when dealing with limited space though it does allow very effective ducting.

1447m89.jpg

^feser 120mm either side of a 38mm san ace, 120mm fans with the hubs removed acting as shrouds either side.

Power supplies capable of running this system will be loud incidentally. I imagine your best shot is the koolance watercooled ones, probably three of them in total. It would be really appealing to run them all from one powerful 12V supply, but sadly the pico psu is not capable of supplying the 3.3 and 5V currents an X58 system requires from a 12V source.
 
WJA96 has 10 or more quads all linked to a domestic rad in his garage.
send him a friendly e-mail to as how he did it. but they all have system fans and in separate cases

I also think it's possable - but 10 mobo's in a box is not a good idea.
I have a Maximus Gene (P45 version) and the MosFet is hot - if you had ten in a box, and didn't want fan noise - you'd need to watercool the cpu,gpu,nb&mosfet as I do.
That a lot of tubing with 10 mobo's and I don't think it would be very impressive.
Infact it will look a bit crap imho.

Better to have 4 max (one on each side of the box)
or all ten set out as 'pictures' on the wall in thin htpc cases (with a window on the lid)
A radient panel or passive rad array can then be places above or below it. (simular to my 'out of reach' build)
 
JonJ678 said:
http://i47.tinypic.com/1447m89.jpg[/img
^feser 120mm either side of a 38mm san ace, 120mm fans with the hubs removed acting as shrouds either side.
[/QUOTE]

That's actually a rather good idea. I may consider doing something similar rather than cracking open the loft in a hunt for some meccano.
Click to expand...
 
ok, so assuming i have let go of the domestic radiator idea, im now looking at 4x (or 6x) thermochill pa140.3 rads to cool the 10 boards.


ill then probably build a pair of loops, each cooling 5 nodes. i can fabricate some custom 5 way splitters to divide the flow to the 5 boards, then recombine before the rads.

anyone got any thoughts on such a system? flow rate? tube diameter? recommended pumps?

i was thinking to use the biggest diameter tubing i can, for the single part of the loop, then split down to some 10mm or even 8mm stuff to the individual components on the boards.

im looking at ek cpu blocks then these kits:

http://www.mips-computer.com/index....tline_MainboardSets_CopperPomBlack.htm?pos=20

for the rest of the hot bits.

psu wise, i was thinking about some Akasa Power Xtreme 1200W jobbies. id use these with splitters to run 2 nodes from each. 1200w could probably run more than 2, but i have to plan for when the guy wants to cram big gpu's in there too.


regarding the comments about heat and putting too many in a box, i dont mind -some- cooling fans, its never gonna be completely silent.. but if youve heard the noise 10 1u renderboxes make when they are going for it (with their dozens of 60mm fans) , He just wants something you can happily have a meeting in the room with, that looks good and doesnt take up too much space.

For me its a nice engineering challenge to design it as small as possible whilst getting the heat out as quietly as possible.
 
You must log in or register to reply here.
Back
Top Bottom