Is that using HW Monitor?
Im using RealTemp and HW Monitor and my Ivy i5 not overclocked using Stulid's paste is idling at 30 degrees roughly. Though I cant see cores 3 and 4 as they dont show up
EDIT:
Got it to work in RealTemp. 33 degrees on all cores max at 5% CPU usage.
Strange, have you tried Core Temp?
Just for the OP reference my idle temps sit around 25-30C on one of the large Phanteks coolers.
I would also expect your to drop a little once the thermal paste beds in.