Do you really believe it would bother me if that was indeed the case? Obviously I'm going the speed limit (probably just over) when it annoys me. But you keep on assuming I'm driving like a granny and won't pull over - and I'll keep going slower if the situation warrants it.
And just exacerbate the problem. It's also amazing how often I've spoken to someone I've been stuck behind who assumed they were doing the limit and weren't.