Having a horse is like a second brain/autopilot as well. They will avoid obstacles, potholes etc. on their own whilst the rider focuses on more important business like cracking skulls.
Yup (although not so much the cracking skulls

)
The Officer can basically concentrate on watching/following someone whilst the horse will generally avoid obstacles pretty much on it's own, or respond to the riders minimal controls/movements
The horses are also very very useful in environments where you go from normal road, to pedestrian areas, or even grass, as unlike pretty much any other form of transport in use, a horse can go up and down kerbs, runs well on everything from grass to pavement or road, and can fit into gaps where a car can't.
I see them regularly in Milton Keynes (they've got a real big beastie*), and the advantages are immediately apparent when you see them around the market, or answering a call (going up over the kerbs quickly to take the shortest route).
Re the horse dung, iirc no horse rider is legally required to clear it up these days, but it can be recommended.
Unlike the muck of dogs and cats it's generally fairly harmless and with little unpleasant odour, that and you're unlikely to accidentally step in it
*I vaguely remember the lass riding it talking to a kid, and mentioning that it was the biggest in the Police service in the UK or something (I'm not short, but this beast was a good bit taller than me, I think the rider could probably comfortably see over transit vans and the like)..