Capacitors charge up by moving tons of electrons from one plate (making it positivley charged) to another plate (making it negativley charged), this can create a very high potential difference (230v in this case) causing electrons to move through your body to the positivley charged plate when you close the circuit.
A pc psu uses a step down transformer which is simply two coils of wire wrapped around an iron core, the coil of wire on one side has a current passing through it from the mains, which causes an electomagnetic field to be induced in the iron, which then generates a much lower voltage in the other coil of wire and a much higher current, which is supplied to the output terminals. Hence why a pc can output 50A through the 12v rails despite the mains being only 13A and the output being such a low voltage. It does not store charge.