Well he can't buy petrol anywhere else or drive anywhere but roads!
You've undermined your argument.
Despite doing everything in their power to follow road courtesy and cause as little disruption as possible to motorists, you claim that the very fact that they are using the road at all when they don't "need" to be makes horse riders selfish.
But your not willing to apply the same logic to a leisure car driver who holds you up. Why? Ultimately it's the same....you could claim he's selfish for being on the roads at all when he had no "need", but you don't.
You've exposed a clear bias that undermines your point.