I maintain my view that the booing only really started or at least became widespread because of him being slow. He has getting jeered, at least from my perspective, not booed. I was largely silent until it became clear he was holding things up.
This may seem like semantics but it's like the difference between when an opposition player gets booked for repeated offences (jeers) or not (boos).
I've already posted in the matchday thread that I could understand his reactions to getting booed in the heat of the moment, if you've got tens of thousands of people having a pop then you might boil over, it's the lack of awareness of the need for urgency that caused the actual booing part of it in the first place. Storming off swearing down the tunnel would have been less annoying to me had he run off the pitch. Trust me, the difference between his number coming up and then 10s later as it became clear he was trudging off was very noticeable, the majority of calls were "come on, get off, hurry up, get on with it" not "i hate you xhaka you are a load of ****"
edit: going a bit OT, to answer the question I don't think we should be changing the manager at the moment. Arsenal are 5th in the league, which is roughly where they should be, doing fine in Europe, and only got knocked out of the league cup on pens away to a team 6pts clear at the top of the table.