I have had a lot of trouble with making eye contact during my therapy. My T said that a lot of trauma survivors have trouble with maintaining eye contact due to shame. I don't know if this is true in your case, but I think it is the reason I can't look at him for periods of time.
For what it's worth,
LJB