Having driven F33 M4s, 435i, 440i and 435d, believe me, I also was convinced that was the case, so I hear you!
I think it's personal and depends on your refernece point..
Jumping from my E46 M3 to an M4 and it's different, but still has that crazy visceral experience, just amped up turbo engine, but definitely an 'experience'..
Jumping in a 440i though and if that's all you know, great, nice engine and you'd be happy, but coming from an M3/M4, it feels so synthetic/numb and dare I say boring.. I've driven the 435i and 440i within a week of each other and just didn't gel with them, hoping the 440i would add a bit more to proceedings, but as soon as I get back in the M3 that same feeling comes back of they are too refined/quiet/linear for a turbo engine.
I then by a freak set of circumstances had a 435d for a few days, driving it to Spa to meet a friend who was racing, it's his cheap continent muncher he uses when touring europe, and so I did a fair whack of mileage over 4 days, roof down etc. I expected to hate it, but was pleasantly surprised, the engine is refined 'enough', still has some noise when revving, but rather than feel a bit too linear/numb, the torque is just that degree more and xDrive just does enough to be a hoot when going over more challenging roads/camber changes full throttle and whilst it moves a bit on all 4 wheels, it keeps forward momentum.
When I got to Spa and mentioned this, immediately a few people chipped in that have driven it and said it's the most surprising car they've driven, and agree, the biggest issue with a 440i is the M4 is just a much better experience on every level.. the 435d has the torque and xDrive that make it 'different'..
Each to their own, everyone tells me the E46 M3 Convertible is wrong on every level, far too much compromise to the handling, yet I enjoy it..