There are some solutions to this issue - in say a high rise development you can have a separate entrance or separate lifts - stick the social housing on the lower floors. In other developments you might have the social housing bits in separate blocks. In the uber expensive developments in West London the developer might do a deal with the local authority to build social housing elsewhere in the borough and contribute to other infrastructure.
Its hardly for no reason at all - a large portion of housing association tenants are unemployed and have issues/don't know how to behave etc.. That isn't to say that all unemployed/low income people in social housing are like that but rather the causal relationship is the other way round - people who are complete ****ups for want of a better word are more likely to be without work and reliant on social housing... whether that is down to drugs, mental issues, criminal records or just the result of bad parenting. Granted there are people with well paid jobs who can be anti social too but it is a bit harder for someone to hold down a well paid job and still play music at 3am on a week night. For some social housing tenants however music at 3am on a week night is fine.
There are plenty of normal decent social housing tenants too but the risk is there. Just as plenty of people wouldn't want to buy a place next door to student accommodation buying a flat next to social housing is simply increasing your chances of being exposed to antisocial behaviour.
I live in a development with 5 normal blocks and 2 social housing blocks... two of the normal blocks next to the social housing blocks also contain social housing. There is residents forum for the 5 normal blocks and unsurprisingly the vast majority of issues with noise etc.. come from the two blocks with the social housing tenants. Fortunately I'm in a fully private block but I get to hear the reports at management company meetings etc.. over the years we've had a guy with 3 pitbulls... who let them run around the communal areas defecating and urinating everywhere, one flat that was occupied by a charity providing young people out of care a fresh start (putting unemployed 18 year olds with behavior problems in their own brand new flat among families/people who have to work is a recipe for disaster), quite a few noise issues from various flats (if you've not got to get up for work in the morning then...), one flat used a spare room as a cannabis factory(people on benefits do sometimes indulge in cash in had work) and another used by an escort.
I'm very glad that I'm in a completely separate block around the corner. Private accommodation increases the chance that your neighbours are functioning adults in full time employment.