The issue here is that people come from different places which results in a divide of opinion amongst those who get involved in this discussion, and it happens every time this topic rears its head. Lots of people don't understand how frustrating cyclists can be (note: not irritating, frustrating, there's a difference) because they don't experience them the way people do in city centers, at least not to the extent of city dwellers. Someone from a nice little village oop norf doesn't understand what a central Londoner goes through in terms of cyclists and this leads to completely differing opinions.
That said, here are why cyclists irritate me, from a Londoner's point of view (and I cycle myself in London, so I see the argument from both sides):
Cyclists can be very inconsiderate of other road users which often leads to very dangerous situations. You can often spot the difference between a cyclist with a driver's licence and one without.
Cyclists can ride around at night with no lights on which can make them hard to see and result in "OH ****" moments, or worse, a collision.
Cyclists can ride four abreast taking up an entire lane and not move over which leads to frustration. See point 1.
Cyclists can jump red lights with impunity which of course sparks the SJW in people regardless of whether it's dangerous or not.
Here in London, half of our roads are being taken away for cyclists which ruins the flow of traffic. Anyone who's been near Elephant and Castle roundabout in the last few months in rush hour will know exactly what I mean.
Boris Johnson and the London Mayoral Committee thought it'd be a great idea to let anyone from any country hope on a bike in one of the world's busiest cities and ride around until their heart's content on any road, with no training or experience. Some of these people may never have ridden a bike before (that's an extreme scenario, but it makes the point), most come from countries where they drive / ride on the other side of the road, some come from countries like Spain or Italy, where the rules of the road are pretty much "I was here first, get out of my way". This just doesn't work but it swept under the carpet because we're so far in to this cuddle-all-the-cyclists mentality that there's no going back without someone ending up with a whole pack of eggs on their face.
People like Cycledan using their helmet cams to deliberately be bell ends and try wind people up for the sake of playing the victim card and make a few bob on Youtube.
//There are other reasons but these are the ones that spring to mind. I understand that it's usually the minority that ruin it for the rest but when it comes to road safety, people get very angry very fast.