OcUK - Stomp Monster Project

Is this project still active, meaning is it still being expaned upon or are there upgrage plans etc?

Additionally, I was wondering if there would be any interest in starting up another project similar to it for F@H?

I've been working on plans for building my own cluster for some time now, and would be interested in working with or leading a project similar in nature.

Just a thought.
.

Magman still has 7 layers (6 working last time I heard). The remaining 10 (or so) were still at OcUK HQ last time I checked, but that was 18 months ago, so who knows since then.

Organising a new project off our own backs would take some doing, but if it looks like it's getting serious, then I'll take it to the Don's Room for consideration.

Berserker said:

Organising a new project off our own backs would take some doing, but if it looks like it's getting serious, then I'll take it to the Don's Room for consideration.

Click to expand...

Mr _Berserker_ - Appreciate the appeal to the Don's < provided there was / is interest >. Here's what I was "Thinking About" from a design standpoint.

--> Master & Slave Node Diskless System < Linux to Save on Lic Costs & Ease of remote administration >
--> Apache Web Sever on the Master for System / Node Performance Data < e.g Ganglia or something similar >
--> Need an IP Address in a DMZ somewhere for access to Stats < heat, loading etc >
--> PSU's could be 1 PSU for every two systems < Saves Costs > does add a bit of heat though.

Base Config Thoughts:
1x 48U Rack < Allows for extensive expansion >
1x 24 Port Switch < Managed for Remote Access would be preferred >
1x APC Master Switch < Also remote Manageable > 1 per 8 nodes.

Processor Considerations: < Power Consumptoipn v.s. Performance Considered>
--> AMD X2's due to power and cost. But the new LGA775 Presler" 2.8GHz have allot to offer, and it has an L2 of 2MB's per Core on a 60nm chip. Also supports SSE3.

Motherboards:
Any mid range Micro-ATX would suffice, as Micro-ATX will allow for 2x nodes per layer in the rack. May be difficult to get a 955X Micro-ATX Motherboard to support the Presler Core chips. Would preferr them to all be the same, but not critical. Would definately want a VGA output onbard.

RAM:
Well, that's not to difficult

PSU:
A descent 450 to 550 Modular PSU would do. The higher (W) the better as we'd be running two board from one PSU, and they'd be more effecient.

CPU Coolers:
A nice ThermalRight XP-120 or SI-120 would do nicely to keep things cool.

Thouhgts, comments, Ideas???
.

I'd definitely go for the P4 option so that she could be pulling QMDs. Need it be so serious with all the rack mounting and jazz? Stompy was just regular ATX mobos in a stack. I know magman's had a tough time maintaining the inner nodes becasue they're hard to access but it seems easy to me to just lay the mobos on a shelf. Put rollers on the shelves so they behave like drawers and you're in business.

With a minimal system I'd imagine a Pressler would pull about 120W at maximum (where it would be most of the time, hopefully). If the nodes were to PXE boot over gigabit LAN there would be need for only two HDDs total (in RAID 1) for all of the nodes. That cuts down power useage as well. If we discovered that backing up 50 MiB per node every half hour overloaded the disk causing the nodes to wait for disk access we could upgrade to some striped hot-swappable disk array.

What sort of future proofing might we do? Do you think that it would be wise to consider the performance capabilities of FAH-on-GPU? They tell us that an x1800 will be able to crunch a Gromacs WU three times as fast as a 3.0 GHz P4.

On an unrelated note I think it's interesting how FAH is picking up so much steam in this forum. If it weren't for the weekly FAH news there would be nearly 75% fewer posts.

Billy -

There was a couple of other considerations I thought about. With the rack specifically, it' makes for a tidy configuraiton / instalaltion, it's easly moved to a more appropriate locaiton < if desired > etc. The cost of a rack is approximately the price of 1.5 nodes, but provides the appropriate enviroment for managing the cluster < wires, switches, power etc etc >. It could also have the names of the contributors plastered on the front glass door

The RAID-1 setup is really a must. A couple of zippy SATA drives would do the job. Disk I/O is rather low once the system is up and running. Would depend on the check-point setting for each instalaltion and the node monitor update frequency. That could be staggered if necessary. We run a 64 node cluster in a similar set up, which has a very high disk I/O rate on some applicaitons, doesnt' seem to hurt performance much, and the disks have survived a couple of years of punding. What are you thinking about with regards to 50BM's per node ? is that due to the Core space requirments?

Another consideration < which simplifies the cluster configuration > is to run a caching DNS server and add in a Simple Router to manage communications to and from stanford or the outside world.

As for future proofing, well, that's tricky, but certainly something to keep in that back of your mind, specifically with the use off GPU's as additional crunchers. I've read allot about the x1800 and it would be a nice upgrade to any system for sure. Have not checked out the price different in motherboard though.
.

KE1HA said:

--> Apache Web Sever on the Master for System / Node Performance Data < e.g Ganglia or something similar >
--> Need an IP Address in a DMZ somewhere for access to Stats < heat, loading etc >

Click to expand...

Use munin. It rocks. check this out. One of the two Team OcUK servers, and it's running F@H, with processing/temp/F@H stats and ability to handle multiple nodes. I think we got that sorted.

Well the technical side of things is relatively easy. The tough part is the organization. Who will take posession of the FAHStompah? Who will feed it, water it, bathe it? How will the money be collected? Who will store the money in between the time that it is collected and the hardware is purchased?

I know Stompy faced these issues but I am unaware of how ya'll overcame them.

Most importantly, what shall we name it?

Indeed. Stompy was (relatively) easy in this respect because it was handled through the OcUK shop - i.e. you could order a share of a layer just like you could any other product. I have serious doubts that could ever happen again, partly because Spie is no longer involved with DC himself, and partly because OcUK is just too big for that now.

These are all good quesitons / comments / ideas, and surely if there's enough interest, we could get this thing off the ground.
.

BillytheImpaler said:

On an unrelated note I think it's interesting how FAH is picking up so much steam in this forum. If it weren't for the weekly FAH news there would be nearly 75% fewer posts.

Click to expand...

Yes, rich does a good job with that.
Sadly, although there seem to be more people joining all the time, we seem to have peaked and then fallen down a bit. Active users has dropped 10, and production has bottomed out. Thankfully, i think were on the up again. I've seen a few people promising to join up with very large numbers of pcs, but nothing seems to have happened.

Reagarding this new folding cluster, shouldn't we be buying cheap cpu's and overclocking them, such as opteron 146's or something. We are the overclockers after all. I'm not particulatly knowledgable on the server front, but if i were building a large number of folding machines, i would:
-Avoid expensive racks and use a diy solution.
-Use the Asrock K8NF4G-SATA2 (754, matx) or 939NF4G-SATA2 (939, matx) with onboard graphics and lan. The K8NF4G-SATA2 is £36 at a competitor, and i found it to be an excellent board.
-Use either cheap sempron 2800+ 754s and overclock them to 3400+ speeds or 939 opteron 144/146's and overclock them. I would not use dual core because they don't overclock so well and are too expensive.
-As noise probably isn't an issue, i wouldn't pay a premium for xp 120. The stock coolers are ok for overclocking apart from the noise, especially the sempron ones.
-Why pay a premium for a modular psu? Ideally these systems would stay on 24/7 and would not be regularly fiddled with.

Just my thoughts.
I assume this will be heavily pimped in the GD for donations?

My thought on the processors was rather simple. With either an X2 or a P4, we'd get nealy 2x the output for the same amount consumption. < if not less, allthough I havent done a formal study on power consumption v.s. ppd on the wu's >.

I know the rack seems overkill, but if we're to convience somebody to fess up 10 to 15 amps of power 24x7 .. they will probably want it to look the part and fit in somwhere appropriate, but ya never know.

As for the Modular PSU, it did cross my mind to run 4 systems < I've found some commercially made power splitters > from one 650W PSU and we don't need all the wire's and crap hanging around to do nothing but disrupt air flow. But point taken on the addiditnoal costs. In a case where your hammering the PSU's quality is important for long term performance & stability.
.

Getting jealous of my foundation's uber-leet P3 rosetta@home cluster are we?

http://www.freeearthfoundation.com/cluster/2/



One question I have....how are we going to afford 30-odd X2s - are they truly the best price/perf?

You've sold me on the rack mount, KE1HA. I've fallen in love with Sparkle/Fortron's PSUs. I've an EPS12V 550 watter in my xeon box and its performance is incredible. It only cost me 50 quid too. Sadly, I don't think that OcUK sells them.

As for X2 vs. P4, won't a similarly ppd performing P4 cost far less than an X2?

Beansprout said:

Getting jealous of my foundation's uber-leet P3 rosetta@home cluster are we?

http://www.freeearthfoundation.com/cluster/2/



One question I have....how are we going to afford 30-odd X2s - are they truly the best price/perf?

Click to expand...

Definately Jellous .!!!

The scope was more along the lines of 16 Physical CPU's but could be more. If the option is the X2 say 3800+ flavor or or the new P4's there about what, 160 ish per processor?

The opteron option say 165 or somehting would be attractive as well, as it's on 939 pin now.

There all just Ideas, but worth exploring mathmatically for the most Bang for the buck.
.

bang for the buck will always go to the QMD-recieving P4.

The power dissipation issues P4s have become an issue when you consider who will take posession of the FAHStompah. If it's going to live in some company's datacenter that's feeding 1000 Xeons and 2 Pebibytes of disk storage, I don't think power is an issue. If it's going into rich99million's garage, it matters.

Where might it go?

KE1HA said:

The scope was more along the lines of 16 Physical CPU's but could be more. If the option is the X2 say 3800+ flavor or or the new P4's there about what, 160 ish per processor?

Click to expand...

That's still £2.5k, and you've not even considered mobos/PSUs/heatsinks/etc yet. Let's say 50 people got involved (a not unreasonable number given the number involved in the original stompmonster), that's £50 each, and probably at least £75 by the time you've added all the other stuff. While some people might be willing to offer that sort of money, I very much doubt you'd get 50 of them.

I'm not trying to blow your idea out of the water here, but I do think you may be setting your aim too high.

BillytheImpaler said:

Where might it go?

Click to expand...

It certainly isn't going in my living room and I don't have a garrage .. the war department would boot me out of the house ... She just doent' understand.. lol.
.

BillytheImpaler said:

Where might it go?

Click to expand...

Ocuk HQ?
If spie is willing to foot the electricity bills as his/ocuk's contributon to the project, that would work very nicely. Otherwise it would have to go in someones garage.

Berserker said:

That's still £2.5k, and you've not even considered mobos/PSUs/heatsinks/etc yet. Let's say 50 people got involved (a not unreasonable number given the number involved in the original stompmonster), that's £50 each, and probably at least £75 by the time you've added all the other stuff. While some people might be willing to offer that sort of money, I very much doubt you'd get 50 of them.

I'm not trying to blow your idea out of the water here, but I do think you may be setting your aim too high.

Click to expand...

Definately a valid point. The desgin / Idea was / is for a system of 16 or so, but, we can start with as little as one master and one slave and gradually add to the system. I personally wouln't have a problem with say twice per year coughing up for a couple of nodes, but everyone personal situation is different, so we'd have to allow for everyone's needs / abilities to participate.

[EDIT]: One thing is for sure, we're not going to solve all the quesitons and potential problems iin one day, so, maybe a list of pros / cons / challanges / oppertunities to excel < management - 101 classes -- see I do remember some things > is in order to hash this out properly.
.

Joe42 said:

Ocuk HQ?
If spie is willing to foot the electricity bills as his/ocuk's contributon to the project, that would work very nicely.

Click to expand...

Not an option I'm afraid. Sorry.