All devices have shortcomings. For example, I loved the iPhone 4 but jailbreaking it and customising made a good phone great.
Adding a customise Rom and tweaks is no different to downloading an App. You want a device to do something extra it didn't stock.
I hold the exact same view when it comes to Jailbreaking the iPhone as it happens. Most people do it so they can get free apps though, whether they want to admit it or not.

I think that changing the ROM/Kernel/etc of the phone is very, very different to installing an app. It really isn't the same thing.
As I said, I'm not saying people shouldn't do that if they want to, just that I personally wouldn't bother with it on the Nexus.
Most people seem to be doing it to fix shortcomings that shouldn't be there in the first place. I'd rather Google worked tirelessly to push out an update to fix these things instead of just leaving it to XDA developers.