Let's be realistic here - Brixton has well above average crime compared to the rest of the country. That's not to say it's a particularly bad part of London though... The main part of Brixton around the tube station etc is pretty safe... venture a bit further out where the estates are etc and you really have to watch yourself, but that's no different to most of South London or even most of London. Camden for instance is well known around the market etc but it is a pretty grimy placed to be honest and there are some properly dodgy areas/estates once you venture a little further out too.
I used to live around the corner from Coldharbour Lane, used to get the train every day from Loughborough Junction to Blackfriars to university. That's probably the dodgiest part of Coldharbour to be honest. At one end you have the cinema in brixton, then about a mile/mile and a half the other end you have a busy junction in Camberwell.