It hugely depends on the situation, essentially I just use common sense and courtesy, if it's someone's private dwelling and it's just a random photo, I'd respect their wishes.. In the same way I respect anyone's wishes to not have their photo taken in public.. If it's some jobs worth telling me I can't take a picture of Buckingham palace, then I politely ignore them.
It's probably legal to defecate through their letterboxes, but oddly, I don't do that either (for the same reasons of common courtesy)..
It is certainly illegal to defaecate through someone's letterbox.
The problem I have with posts like this is it is equating taking a photo of a house to vandalism.
Taking a photo of someone's house is actually no different looking at their house with your own eyes, which is why it is legal. If you uphold the request of the irrational homeowner then really you couldn't even walk down their street without being blindfolded.And that is exactly why their stance is so irrational.