If I am talking directly to someone, I don't have trouble looking them in the eye. I just don't hold it very long. Besides the feeling of unworthiness which can be related to how attractive I think the world perceives the other person (yes, the ulgier the person is, the longer I can maintain eye contact), I also understand it is rude to stare. So it's wrong to have no eye contact and wrong to have too much eye contact. As a result, I try to look at someone in the eye until I sense they are looking back at my eyes, then I look away for awhile, then I let my eyes wander back to theirs and repeat the process.