100% this. Was on a 4hr flight recently where we'd paid for extra leg room. I had window seat, with my wife in the middle seat of a row of three. The other seat was occupied by someone who couldn't actually fit in the seat without encroaching in to the aisle and my wife's seat. Made it incredibly uncomfortable for us for the whole flight. If you can't physically fit within the boundaries of the seat, and I don't care about the reason (could be a 6ft5 body builder with 35" thighs or just someone who pretty much lives in Greggs), you should have to buy the seat next to you.
Thats the funny thing. If you are huge because of muscles, they WILL tell you your too big. But if your Jabba the Hutt, they won't because they don't want to seem un-PC :/