I've just seen that, according to the manual, the power switch is supposed to go under the GPU which is impossible.
I'll take the GPU and put it on a different slot.
[EDIT] Ok, I am currently trying to screw the GPU back in place to no avail.
Its hard to get it to stay in place and screw it in. The PCI-E clips broke off each one too which isn't helpful.