Maybe I missed it but have you tried the gpu in your 2nd pc.. as to me sounds like a hardware fault with it hanging while turning off.. before when a had a hard drive that was starting to go faulty . It made the pc hang for a while when booting up... also not sure if you have other hd's connected , if so just try it with the 1 hard drive. Good luck !!
If it was the SSD then I should get signal into the monitor asking for a booting device when trying without any hard drive, so I think it's safe to say that the SSD is not the problem here. And for the GPU is the same, without it the computer should boot fine, and in fact it does not. It does the same (nothing) with or without the GPU and the SSD