Windows 32bit limits each application to 2Gb memory space, and by default the physical memory is divided into 2gb userspace, and 2gb kernel (Although the /3gb boot hack switches to 3Gb userspace, and 1gb kernel).
Anyway, a limitation of 32bit windows is any 1 application cannot access more than 2gb of physical ram, above 2gb pages will be put into the swapfile instead.
64 bit windows actually has the same 2gb limitation, however with both 32bit and 64bit software there is a complier option which enables large memory support. For 32bit applications on 64bit windows that option extends access to physical memory to the full 4gb, for 64bit windows the limit is set by the version of windows. Vista64 ultimate has a whopping 128gb limit, and even that is a "software" limitation.
4GB on 32bit windows is pretty limited use really, although not completely useless the 2GB limit I mentioned is per application, so say you have 3GB memory after the graphics cards ram, and other bits and bobs are mapped in, windows itself + antivirus/firewall etc will take their normal few hundred meg of ram, and your game will get a full 2gb all to itself. Clearly better than 2Gb shared between game, os, and various other processes.
Interestingly, 32bit applications compiled with the large memory option, will run just fine on 32bit windows, but with the standard 32bit limitations, so there is disadvantage to the software developers to enable the additional memory support on 64 bit systems, while remaining compatible with Windows 32bit

. (Compiler flag for Visual C++ = /largeaddressaware). Im not sure if its possible to see which flags were set during build though.