I've tested the RAM multiple times and it passes memtest86, Windows XP was rock solid with this same setup so it leads me to think it's something vista related.
And your bias against the OS I'm sure has nothing to do with that theory.
Could be the USB drivers. Have you tried reinstalling chipset drivers?