Yeah, I do think the board does have a little to do with it. Not saying you will be getting a huge boost if you changed to the board I have, but there is probably a good chance you would at least get a 100mhz all core boost.
I did Highlight - Right click Google Search the board in your signature, i like it, a new board has been on my mind even before i replaced the Ryzen 1600 but the delay of B550 boards has put a halt to that idea, i find it difficult to stomach £200+ on a board for a £180 CPU.
I've even considered going high end B450 or mid X470 with the prices on those now being quite reasonable but i know if i did that B550's will land soon after and i would kick myself.
I'll wait a bit more...