Is it the 2500 or the 2500K? The stock fan is fine up to 4ghz but if you have the non K version you can only do a limited overclock, as shown in Davefran's post.
Also, you need a P67 or Z68 motherboard to otherclock, otherwise you're pretty much stuck on stock speeds.