IEEE compliance is the big thing. It states all the sizes and the behaviours for exceptions.
More indepth here:
http://en.wikipedia.org/wiki/IEEE_754-2008
Basically your gaming texture shaders use floating point numbers for the colours on the texture. The shader program changes those colours depending on light etc, and then the GPU transforms them before rendering to the frame buffer (screen). These texture pixel colours are stored as 4 floating point numbers.
Historically, for speed they'd be 16bit or 24bit without any error handling (divide by zero for example) that a normal CPU would perform.
So if you remember all the fights about 16bit vs 24bit vs 32bit shaders then that's where the 'bitness' comes in.
A shader is a very simple, very specialised and highly efficient number processor. It's only got a few operations compared to a CPU. However it when it does a calculation it does the four channels (R/G/B/A) of the pixel in one go. One such operation is multiply and add, Result:=A*B+C. This is the most common maths matrix functions multiply and add... we'll come back to matrices later..
Now a GPU uses a massive number of shaders to process each texture pixel in parallel. So a texture may be 4096x4096 pixels where each pixel has red, green, blue and alpha values stored as floating point numbers.
So if a GPU has 512 shaders then it can really process a lot of numbers at one go!
In GPGPU the technique is to abuse this number crunching power for highly mathematic purposes.
What you do is load your numbers into the RGBA for each pixel in the texture and then load a shader with the processing you want todo. You then render the rectangle texture to the framebuffer without displaying to screen. This frame buffer holds all your results that you then read back from the GPU into main memory to get the results back.
The result is that you can then use your 512 shaders to process 4096x4096x4 floating point numbers in one go! That's 67,108,864 floating point numbers in one go!*
Compare that to you CPU SSE instructions that do 8 numbers in one go!
Now with games you can use those that power todo proper collision detection, physics calculations etc. The games only require a low precision as they don't have to be accurate thus faster single point is acceptable.
Now in super computing and modelling, accuracy and precision (yes they're two different things) are important. A minor error will, as the model progresses increase in size causing problems. Now models usually have error calculations too but the behaviour of the number processing needs to be predictable. Hence the IEEE behaviour compliance is required.
Supercomputing tends to be high precision too - this means the numbers are 64 or even 128bits in size. However there are techniques that can be used (error calculations etc) that can be used to reduce the precision required as a trade off for speed to get results back quickly - often these are small runs on smaller subsets of data and will be done by the coder's PC overnight or a dedicated PC run for a week for a test.
You have to remember supercomputing performs trillions of matrix multiplications and additions/subtractions on terabytes of data and the machines that do this are hideously expensive so time on them is not available for 'testing' and the cost for a "run" is also high.
Having 64 bit IEEE double precision floating point numbers allows Fermi to target this area of supercomputing. Areas such as analysis of readings from ground or ship based radar to detect oil for example are all areas where a quick estimation on the ship allows the company todo make many quick checks rather than having to wait for supercomputer time todo their calculations.
Ok, a bit war and peace but I hope that helps.
* yes this is simplified but it shows the spirit of the use rather than get into complexities of SPMD, shader groups etc. Also "a go" means one execution command by the CPU to the GPU.. thus 512 or less are actually in parallel depending on the shader program however the GPU will still finish all the texture pixels before completing the shader program (thus all 67M numbers).