a headphone socket has already been amped, by the internal amp.
otherwise it would be a line out and need amping.
effectively by using an amp on a signal which has already been amped by a cheap internal phone amp means you are just amping a crap signal and effectively making it even crapper.
there should be much more distortion and hissing, etc by amping a headphone socket than just using the headphone socket on it's own.
basically as a rule of thumb, you never ever amp a headphone out line, if you want decent SQ, you only amp line out signals.
your setup is like this
sound > crap amp > good amp > headphones
basically your just amplifying the crap amp
on a line out on an ipod
sound > good amp > headphones
you bypass the crap amp and therefore you amplify only using the good amp meaning much higher sound quality.
amping a signal twice destroys the sound quality, otherwise every DJ, or home cinema setup or car audio setup would run 2 cheap amps instead of 1 expensive one, think about it.