OK, those voltages are well within the ATX spec. It could just be a corrupted Windows install causing your issues.
Likewise, it could be a motherboard issue. The random reboots do point towards the PSU, but as all the voltages are OK, it's hard to say if that's the issue or not.