I've got no issue with people making mistakes. That's human nature. People will stall, people will crash, people will find themselves in the wrong lane, people will miss the occasional filter light.
What aggravates me is this habitual poor driving. That woman who just pulled out right in front of me without looking? You can bet your salary that she's also going to:
1) Not accelerate to speed
2) Ignore speed limits and drive at her own pace
3) Unduly hesitate at each junction
The guy in front who's not accelerating up the motorway slip road, leaving you pinned in behind him? He's also going merge at the top without looking, launch it into the middle lane without checking if anyone is behind him, and then be unable to maintain anything like a consistent speed on the motorway.
People just don't care about driving. It's something they have to go through to get from one place to another and generally they take zero pride or effort in the process.