I've not really seen that, for example the smallest room in my townhouse is bigger than two of the bedrooms in my dad's Victorian semi. My old childhood bedroom is absolutely tiny, less than half the size of the smallest bedroom in this house.
Similarly, in our old Victorian semi, the third bedroom was just a single room, maybe half the size of our smallest room.
Townhouses just make a lot of sense to me from a logic/efficiency perspective in a crowded nation with expensive land, more floor area for a given land footprint, so it's a more efficient use of space. Similar to how in the big cities you get a lot flats, you can't just have hordes of detached bungalows and stuff.
That's not to say I wouldn't prefer a 'flatter' home with less need to go up and down stairs, but for a given footprint, I'd obviously want as many decent sized rooms as possible. I wouldn't want to pay more money for less floor area.
The biggest downside with townhouses for me is that they perhaps lack ground floor space, really I want at least 100m^2 on the ground floor which is quite rare in townhouses. Things like a study tend to get pushed upstairs.