F@H - Linux + SMP + NV-GPU2

Associate
Joined
3 Jul 2004
Posts
866
Location
Helena, Montana
I've been rooting through many threads on various forums & wiki's, and it seems there's a viable installation path for Linux 64-Bit SMP + WinDoze GPU2 using wine, CUDA 2.0 and a custom Driver Wrapper.

Pro's:
--> Linux SMP Goodness
--> Nvidia Multi-GPU Goodness
--> ATI GPU Goodness (TBA, it's in the works from what I've read)
--> Headless Operation (No Monitors, Mouse, KB)
--> No WinDoze Lic. or the usual whooo's of WinDoze XP / Vista.

PPD Nominal Estimates:
--> Q6600 @ 3.0Ghz + 2x 8800 GTS (G92) Stock => 15K
--> I've seen data on a Quad with 4x GPU cards, it's shocking the PPD.

Con's / Concerns:
--> Requires a Custom Driver Wrapper = Done.
--> Native Linux Install, so you loose your WinDoze, unless it's a VM.
--> Have not seen a VMWare Install version, not to say it's not happening. If possible, it makes OC'ing the GPU's a breeze.
--> Takes some configuring to negate the need to X11 to be installed. Not a major issue.
--> Not Fully Supported by the Pande Group, but it's not being rejected.

So, where does that leave us. We need a rock solid Method / Guide for installing this beast (maybe add it to SirusB's F@H SMP Client Guide ?? ). I'm not a Linux GURU, but I can follow directions well, test and provide feedback.

Any Takers ??
.
 
It's a reality. Through the deticated work of many people ( too many to ccredit here), I along with many other folks are running Linux SMP + GPU's and so can you, it's dead easy (once you've cracked the first one)

Here's my current Setup:
CPU: Q6600 My grandmothers wheel chair rolls faster than that thing will clock (Duff Cores)
MOB: GA-P35-DS3L, was in the garage collecting spiders webs
RAM: OZC PC-6400, Old Junk I found in the parts bin
PSU: ToughPower 650, with 2x bad 12v Rails.. :eek:
HDD: 20GB Segate, bottom of the barrel crude
CAS: Peice of Wood: WxDxH 24"x12Xx1/2" Looks good though. I used Velcro to bolt things down ... LOL ...
GPU: New EVGA 9800 GT (Only descent part in the thing)
OSS: Ub-8.10 Server Edition, CLI onlly, no GUI
NIC: Netgear 311v3 .. Used ndiswrapper , the sys and inf files only, up and running in seconds.

Here's the Best Part: (1x SMP + 1x GPU, from mostly trash bin material):

-- Q6600-3-1 --
Min. Time / Frame : 10mn 57s - 2524.93 ppd
Avg. Time / Frame : 10mn 57s - 2524.93 ppd
Cur. Time / Frame : 10mn 59s - 2517.27 ppd
R3F. Time / Frame : 10mn 59s - 2517.27 ppd
Eff. Time / Frame : 11mn 04s - 2498.31 ppd

-- Q6600-3-GP1 --
Min. Time / Frame : 1mn 24s - 4937.14 ppd
Avg. Time / Frame : 1mn 24s - 4937.14 ppd
Cur. Time / Frame : 1mn 24s - 4937.14 ppd
R3F. Time / Frame : 1mn 24s - 4937.14 ppd
Eff. Time / Frame : 19mn 11s - 4937.14 ppd

Total PPD: => 7500 PPD .. I can live with that.

I've not even come close to maxing out the cores. Load Averaves are 3.36 / 3.42 / 2.58 ... the 2.58 will go up to mid 3's in a day or so.

So, with 2x GPU's and 1x SMP, your looking at a minimum of 12K PPD.
.
 
Last edited:
After I figured out I had HW problems, things went smoothly. The install, assuming you already have Linux-Server CLI installed takes about 10 minutes.

I suspect this will be automated in the not to distant future, as the benifits for Stanford are pretty healthy from a production standpoint.

There are 2.25 Million Widows cpu's, 345,000 Linux cpu's, and 60,000 GPU clients (ATI + NV). If they pull just 10% of the linux Clients, they would double the number of high output GPU clients.

That's not the best part. Even with 2.5 Windows cores, the TFLOPS pale in comparison to NV which is 1754 TFLOPS, tied with PS3 at ~ 1753 TFLOPS.

The TFLOP numbers are staggering if the adoption of GPU folding rolls into mainstream applications.

I have a side project I'm been sketching up using mini-ITX boards that have a x16 PCIE slot. Wont give away the details yet, but it's goning to be cool.
.




.
 
Last edited:
Looking forward to it KE1HA! I always thought it would be cool if there was an (intel) atom based motherboard that had a PCI-e x16 slot, but i couldnt find one... Would really cut the power consumption and keep the points up from the a gpu.
I dont know about an Atom based / VIA board, but there are mini-ITX baords that support C2D, they deffinately have x4 or above PCIE slots, that's all for now :D
 
That's cool stuff, but 2500ppd is pretty low for 4-core SMP on Linux. Is it running an a1 core WU or a2?

I tried to get this working a little while ago. The Q6600 at 3Ghz was doing nearly 6000ppd from one SMP instance on a2 core, but I gave up because I couldn't even get the nV driver to install without killing the X server, let alone the GPU client. Good job on getting it running! Automation would be nice as the process for setting up the custom CUDA wrapper is quite involved... then again, most people who are running Linux don't mind a bit of tinkering.
SMP has actually gone up a good bit, mid 3500 range. Will post the data later this eveing, as that will be a full 24 hours of crunching on both.
 
Rig-1
1x 9800 GT Stock
1x SMP CPU at 2.7 ish
Total PPD => 8,390

-- Q6600-3-1-SMP --
Min. Time / Frame : 8mn 43s - 3171.85 ppd
Avg. Time / Frame : 8mn 48s - 3141.82 ppd
Cur. Time / Frame : 9mn 01s - 3066.32 ppd
R3F. Time / Frame : 8mn 53s - 3112.35 ppd
Eff. Time / Frame : 8mn 57s - 3089.16 ppd

-- Q6600-3-GP1 --
Min. Time / Frame : 1mn 19s - 5249.62 ppd
Avg. Time / Frame : 1mn 25s - 4879.06 ppd
Cur. Time / Frame : 1mn 19s - 5249.62 ppd
R3F. Time / Frame : 1mn 19s - 5249.62 ppd
Eff. Time / Frame : 1mn 19s - 5249.62 ppd

---------------------------------------------

Rig-2
2x 9800 GT Stock
1x SMP CPU at Stock 2.4Ghz
Total PPD => 12,045

-- Q6600-1-1 --
Min. Time / Frame : 10mn 22s - 2667.01 ppd
Avg. Time / Frame : 10mn 27s - 2645.74 ppd
Cur. Time / Frame : 10mn 22s - 2667.01 ppd
R3F. Time / Frame : 10mn 22s - 2667.01 ppd
Eff. Time / Frame : 10mn 21s - 2667.01 ppd

-- Q6600-1-GP1 --
Min. Time / Frame : 1mn 23s - 4996.63 ppd
Avg. Time / Frame : 1mn 23s - 4996.63 ppd
Cur. Time / Frame : 1mn 23s - 4996.63 ppd
R3F. Time / Frame : 1mn 23s - 4996.63 ppd
Eff. Time / Frame : 1mn 24s - 4937.14 ppd


-- Q6600-1-GP2 --
Min. Time / Frame : 1mn 23s - 4996.63 ppd
Avg. Time / Frame : 1mn 23s - 4996.63 ppd
Cur. Time / Frame : 1mn 23s - 4996.63 ppd
R3F. Time / Frame : 1mn 23s - 4996.63 ppd
Eff. Time / Frame : 1mn 23s - 4996.63 ppd
---------------------------------------------

Im just happy I dont have to mess with Dummy Loads or Monitors at all with these things now :D
 
Back
Top Bottom