I've been noticing it more recently, I put this down to playing on my PS3 and then switching to the 360, I really notice the noise difference then.
Exactly the same for me mate, the sound from the 360 has always bugged me, I've owned a few 360's and they've all been noisy, usually I've just put up with it even though it was distracting, but then when I brought my PS3 the noise from the 360 became too apparent, I would be playing a game on my PS3 with not even a whisper of noise with my TV on low (at night I don't want to wake people), then immediately after I switch the 360 on and I was like

, absolutely unacceptable in my eyes, my PC which is a hell of allot more powerful than the 360 is no where near the volume of the the 360 and it has about 8 fans in it, in fact thinking about it, the 360 was the most noisy electrical component I've ever owned bar the washing machine on the spin cycle,
I'm not trying to put the 360 down on purpose, partly why I'm mad is because I love the games on the 360 it's just so annoying that the hardware so sooooo poor, basically playing a game on the 360 comes with so much mental baggage, not only do you have to endure the jet engine noise of the console itself, then you have the noise from the PSU, oh, then you have the noise from the optical drive, oh, then you constantly have worry about getting the red ring of death, even if a menu takes a few seconds longer to appear in your mind your think "well is this it then" and you sit there waiting for the red ring to appear, when it doesn't you rejoice that you console has lasted another gaming session, oh, then on top of that you worry about the drive scratching your games that you've brought out of your hard earned cash, the list just goes on, it's a complete joke, how it even got off the factory line I just don't get seriously.