I live in the biggest council estate in Portsmouth - feral pre-teens, wannabe roadmen, smackrats and genuinely nasty runts everywhere, but only once had an issue when we first moved in: installing security cameras on each corner of the house, which upset the balaclava'd Sauron-rider doing his Friday afternoon Columbian marching powder deliveries to the constant queue of MSRT van drivers pulling up by my house.
Dealt with it by using humour and making it crystal clear that my only interest in life is keeping my family safe and secure. What they do amongst themselves, how they make a living or what they sniff to get through the weekend, doesn't interest me whatsoever. "Not my monkeys, not my circus", etc.
An unexpected perk of him using my front yard for his handovers is that the other toerags stay away from "his" corner.