Lets not forget about companies that use website filtering software. Theres no guarantee that public sector or some companies will see your site in work time anyway. I know as I speak of experience. So maybe that 5% you speak of, could i reality be 2% due to website blocking software installed on company servers.
For the sake of including a different style sheet, even if it is 0.000000000000000001%, there's no argument. An extra style sheet is going in.
Like I said, it is *your* site that is losing out. A marginally less "flashy" site for IE6 is all it needs...