Yea...PD seem to do well only because of its default high clock...if you look at it from the view that it only got around 10-15% of overclocking headroom left, comparing to 39%-45% of Ivy...
From what I've seen so far the overclocking headroom on Piledriver is a bit higher than that. Most people on the 8320 seem to be getting 20% plus.
To the OP, it's not worth changing CPU's as the i5 is a great gaming CPU but then almost all current gen CPU's are (The graphics card is a bigger factor really). If you ended up moving into rendering more then Piledriver would be worth it but stick with what you've got for now.