Awesome Supercomputer Build

[Rooster212]

Great video of a build of a supercomputer in Switzerland

http://youtu.be/k32Bwy9rha4


Pretty cool. Especially with specs like this.

CPUs: 384 Intel Xeon E5-2660, 3072 cores total.
192 Main Boards
12.3TB of RAM
192 OCZ 128GB Vertex 4 SSDS (24.6TB space total)

More info on it here.

http://www.itp.uzh.ch/~stadel/doku.php?id=zbox:zbox4

 

[Darkans]

that is insane, I always wondered how they built those, thanks for the post

the cost was roughly £497,400, madness

 

[Tattysnuc]

Wow! Imagine the folding power of that beauty!

 

[Space Monkey]

Imagine switching it on and you get a memory error, who gets to memtest that sucker!

 

[StonedPenguin]

Power usage (full load): 44 kW

 

[Rooster212]

Imagine switching it on and you get a memory error, who gets to memtest that sucker!

3 of their CPUs didn't work when they switched it on, i'm sure they had a whale of a time troubleshooting!

 

[Deceptor]

It has more RAM (12TB) than I have in actual HDDs/SSDs combined!

 

[Rooster212]

It has more RAM (12TB) than I have in actual HDDs/SSDs combined!

I think that 99.99% of people would say the same thing

 

[olivier renault]

half a million pound, about 60 teraflops. Not bad.

 

[Fire_fly]

half a million pound, about 60 teraflops. Not bad.

But will it play Crysis at 60FPS?

 

[AshD93]

Crysis on medium maybe?

Sorry, it had to be said.

Amazing build vid. Really interesting to see it.

EDIT: FIREFLY!!!

 

[Fire_fly]

Hahahaha

 

[Dr Delphi]

Nice to see they have put SSD's in each node, this was something I was talking to my cluster manager about a year ago, so there was somewhere with high throughput to write the data which then could be brought back together into one file when you want to end the program with a simple bash script. Needless to say back then it was a little too expensive to do that to the university supercomputer!

 

[pcAnywhere]

All that building and no anti-static wrist bands

 

[coops1992]

that is insane! how do they link all the systems together?

are they just used as a server and VM's

 

[Dr Delphi]

that is insane! how do they link all the systems together?

are they just used as a server and VM's

They are linked together using low latency fibre optic based infiniband system (that's usually the most expensive part of building a Beowolf cluster). This allows each node to talk to each other. Usually they are used for massively parallel programs, whereby some area of the simulation space can be partitioned and passed off to other nodes/processors/cores. For example when I was working on my PhD, I had to use a supercomputer in Italy called Cineca to calculate ice with a CFC on the surface from an electron and quantum framework. I used well over 1000 processors, and had a realtime calculation length of a month or so, my peak RAM usage was over 20TB, and some of the simulation files were of the order of 500GB!

 

[coops1992]

20TB of ram is mind blowing!

id love to be involved in building something like this one day! will never happen in my line of work though

 

[NightSt@lk3r]

Thats going to be a lot of warranty returns on OCZ SSD's

 

[Dr Delphi]

20TB of ram is mind blowing!

id love to be involved in building something like this one day! will never happen in my line of work though

It was a big calculation - got me a nice paper though in the end! I'd love to be a supercomputer manager, but its not really the area I got into in the end. You can build a cluster using ethernet though, so you could always build one in your garage - if you do and you need it testing, I can always run some of my programs, they'll use 100% of all the cores for as long as you want!!

 

[coops1992]

i dont have the money nor the time to do a project like that... Public sector jobs seam to be work more hours for less money now-a-days