It isn't the speaker... it's just the audio hardware in general. I can hear the slight hissing clearly through earphones in the period after I pause a song. I suspect it occurs through the speaker as well, although it's so quiet even through earphones that I don't know how anyone could possibly be bothered by it!
[edit] Yes, I can hear it through the speaker as well. But only when the phone is literally an inch or less from my ear, and only for a few seconds after it plays a sound. Never noticed that in four years of owning iPhones, and probably never would unless I deliberately looked for it!