Try your old PSU? Definitely noise coming from the PSU?
It’s absolutely definitely the PSU.
The recording was taken with the PSU removed from the case.
It’s nothing to do with the PSU fan either. The fan spins up fine when the computer boots, with no adverse noises whatsoever. Then stops spinning as it’s supposed to because the computer is idle.
No idea if this is relevant, but when I power off the PC there’s a real ‘ping’ from the PSU as it shuts off...