Load not bad, really, idle is really bad. Either something is forcing the CPU to boost/running processes or bad contact (which I don't believe, as load temperatures would cause thermal throttle.
It's a game of finding what works best for your CPU.
Just for comparison, different Ryzen Gen, but my best voltage for temperatures without affecting much of the boost is with PBO enabled and voltage offset to - 0.2. Using - 0.15 my temperatures during stress test increases about 6C. Using an offset of -0.1, the temperature increases up to 12C compared to the -0.2 offset.
Spend few hours this afternoon and ended up using the offset of -0.1, as unless running stress test the CPU would max at mid 50's to 60C (games or 3D Mark). Even during stress test would max at 72C, which I found acceptable. Reason for opting for -0.1 is because I noticed the single core boost would happen more often and for longer. 300+ on Fire Strike, and as gaming is the more demanding task I use the PC for, the CPU would be at 60C or less.
I would recommend some play with voltage, nothing too aggressive, just to get a feel and try to tame the temperatures a bit.
But the first call is checking your power plan on Windows and make sure the minimum processor state isn't set at 90, 95 or 100%. Mine I have set to 5%.