It's not simply a matter of the PSU supplying power. It has to provide the right voltages to the right places in order for your PC to boot. It only takes a minor fault to produce the symptoms you describe.
I'm not saying its 100% going to be the PSU at fault, but it's the most likely, and it's the first thing you should rule out.