Well you learn something new every day but it doesn't change the argument.
I don't assume every person walking down the street is a burglar or pedophile and someone living in a nice house should either if they see someone casually taking photo of the hay fields who turns around to snap at their Tudor cottage.
Anyway, this is just going round in circles.
The problem with your argument is that you refuse to see both sides. I don't assume everyone I see is an axe murderer but odd behaviour will always draw my attention. For some people having someone stop to take a picture of their house isn't normal and will draw comment. I'd certainly ask someone what they were doing if they took a picture of my house, and I'd most likely shout from a window rather than lose sight of them.
It's all making a mountain out of a mole hill to be honest.