I run XP Pro x64 with 4GB RAM and don't use a pagefile, and have no problems. If you have some system crashes, then you may not get the crash dump data without any pagefile, so a minimum sized one is probably wise. If you do have one, you can get better performance with a fixed sized one (ie. minimum size == maximum size), as Windows will then not have to increase the size dynamically, which can slow it down a bit. The downside of doing this is that you will lose this amount of space on the hard drive, even if it's not needed.
As others have said, you'll get better performance having the pagefile on a different physical drive from the one the OS is installed on.
EDIT: I should probably add that I don't run any games on my system.