I went from F21 M135i to F80 M3 and have driven both extensively on the road (both >60k miles around the UK and Europe) and enough time on track in both to believe I can confidently call the differences between an "M Lite" (I hate the term - there's nothing "M" about them!) and a full-fat M car.
On the road, unless you're REALLY pressing on there's not a lot between them. Both are exceptionally good at overtaking, both feel rear driven with uncorrupted steering (maybe this differs with the newer generation of xdrive models), both are comfortable for long journeys but the normal car probably more so than the M car - they are much softer. However at very high speeds (>120mph) on the autobahn there's a very noticeable difference between the cars. The normal car begins to feel very floaty and disconnected from the road, especially at the back end. It's easily unsettled by bumps and uneven road surface. At these speeds the M3 feels, if anything, even better like it wants you to go faster. It really hunkers down and feels absolutely solid, even at >150mph and inspires confidence in the platform. Missing an LSD is very apparent if accelerating hard out of a bend or junction. The power and torque of the engine easily overwhelms the rear tyres. The LSD in the M is excellent giving substantial traction improvement in these moments and a much more planted feel to the car. Again, maybe the LSD in the newer M340i addresses this. Finally, we have the front end, and this is for me possibly the biggest difference between them. On the road the M3 feels like the front end will grip FOREVER - it feels abolutely solid and it will go wherever you point it. This is very much not the case with the M Lite which is very biased toward understeer. This difference becomes even more pronounced on track.
On track the cars are worlds apart. The high speed stability, the "connectedness" of the chassis to the tarmac, the front end especially and just the sheer traction and controlability of the M car are in a different league to the normal variant. The floatiness of the standard car becomes really noticeable with a sloppy feeling to the chassis, the front end washes out quickly and with the missing LSD you struggle to use the rear drive and torque to steer it - you have to wait ages to get back on the power. The M car by contrast feels sharp and precise, and keeps whispering "more, MORE, MOOOORE!" in your ear.
As I said previously, none of these points make the M lites a bad car. They are great cars, with superb engines that have way more power than you need on the road. But they are not M cars.
To tie this in with the Porsche mentions...the M3 is also not a 911. I recently had the pleasure of several hours on track in a 911 C2S at Porsche Silverstone and I was amazed (given how similar on paper it is to the M3) how good it felt on track. Compared to the M3's bowie knife approach the 911 felt like a scalpel. It was noticeably lighter, nimbler, the brakes felts incredible (standard steel discs...nothing fancy) and the front end was again on a different level. I can see why a lot of people go from an M car to a 911 - it's the next logical step.
But again, a 911 isn't an M3 (and TBH, neither car pretends to do what the other does).