If you mean 'rubbing people up the wrong way' as in people see him for the lying, cheating disingenuous person he really is, then I agree. But it wasn't soon enough, he should have never even made it to a second election. But it's a funny old world, politics.
Ken will be back, somewhere as something, maybe even an independent after Boris leaves the scene.
