Generally speaking I don't make eye contact with people - if I do, it is usually a conscious effort on my part (say in an interview situation). This has been commented on occasionally.
Part of the reason for this is that because I'm visually impaired, I don't use visual signals all that much myself. In fact sometimes at work If I'm talking to someone off to my side I'll only turn to face them slightly, definitely not looking at their eyes/face. I tend to be influenced more by audio cues (tone of voice, phrases used) etc - in fact I can often recognise people coming by the sound of their walk.
It's also possible that I have something in my mental makeup which causes me to be this way - my sister has Asperger's and I do share some traits with her. Subconciously I think my reaction is "if I stare at this person for too long, I'm gonna appear weird".