I fully agree with you there.
I think the "majority" of people are relatively centrist who lean a little bit left or a little bit right but generally stick nearer to centre than to the extremes and, because they're fairly centralist, they just want everyone to get along without any fuss, follow the rules like they do and just get on with their life, which is one reason why I think Labour, who adopted so many fringe beliefs as a party, did so poorly in the election and yet didn't see it coming, despite all the evidence being laid out simply for anyone to follow.
Worse for me is that over the past decade a very small but very vocal fringe has appeared on both sides, with the right fringe being shutdown very quickly, loudly and publicly, forcing it back into hiding to lick it's wounds. Yet in comparison the left fringe has continued to shout their version of bigoted "hate" speech unabated, celebrated even, with anyone who stands up to them risking not just physical violence (a tool used by both sides) but nowadays risking financial ruin too which could be considered much worse in the eyes of the average person.
Sadly whilst I don't think for a second that Mr & Mrs Average "little bit left/right" Centralists like myself agrees with the vast majority of what this extreme fringes decree we are, maybe understandably having seen what happens to people to speak out and just with them just wanting a quiet life, deciding to keep quiet, to not challenge the fringes because of the perceived "risk". Meanwhile the very obvious hypocrisy of how different "sides" are dealt with will just continue to ferment further division rather than inclusion*, increasing the levels of anger in people until there's a tipping point and eventually a "strong" authoritarian Political party (probably Right wing) which promises to get tough on the fringes gets voted in, and then we're all in the poop.
But thats just me guessing and being a little cynical about life I suppose.
* - As an example look at recent events in the US where inclusion is now being ignored and segregation/division is being brought back by the Left fringe - actors can only play characters with the same skin colour/gender/sexual orientation etc as the actor, different national anthems for Black/White, majority "white" protests banned and/or negatively reported on but majority "black" protests allowed and praised as "peaceful" when thousands of businesses have been damaged (1500+ in one city alone) costing billions, California repealing is Civil Rights act so that the state will be able to discriminate against a person based on their skin colour once more etc etc - None of that is "inclusive" and all of it just adds to division.