OK, first off: Don't use the automatic overclock, it's really not good.
Second: I imagine you're using stock cooling which isn't the best ever, so if you want to get above about 3.5Ghz, get something like an Arctic Freezer 4.
Third: That Nanya RAM isn't good for overclocking, so it the BIOS find the option that says "DDR frequency" and set it to either 266 or 333. That should eliminate the RAM as a problem.
Fourth, and finally: If it's still unstable when running Prime95 or some such stress tester, raise the Vcore a notch, see if it's stable. If it's not, increase the volts again. If it is, increase the fsb until it becomes unstable again before increasing the volts some more.
I reccommend not going over 1.5Vcore for everyday use.
EDIT: If you don't have Prime95, get it
HERE. It's SP2004 which is Prime95 but easier to use.