He must be very special to you, cause u care about him a lot, even when u know that he has problems. I envy him; what I wish is to have someone care about me a lot like u care about him

People who have SP like us often feel very lonely, so we will feel happy when there is someone that can sympathize with us and talk to us whenever we need. So I can tell that the guy u talking about really like u.
If I was him, I think I certainly fell for u; but I would prefer keeping you as a friend. Cause I know it would be hard for me to do anything for u and would cause u only troubles. Loving a guy with SP means that u may have to sacrifice many things. People with SP are often afraid of going outside so its really hard for us to go on a date, or even smallest daily activities. I think he like u, and thats the reason he doesnt want to hurt u, so he keeps distance from u. Thats only my opinion, based on my own situation.
However I think that u re a very determined girl, and u may have the power to change his thought. U have to take the initiative, cause shy guys like us will probably never do it. Try to talk much to him and listen to what he says. Try to ask him what he feels inside and tell him that u will listen and sympathize with everything he says. Ill feel very comfortable when I have the chance to confide my Social Phobia to someone. And we often feel depressed so try to understand and lighten him up whenever he does. Ive heard that love can somtimes cure SP

Wish u all the best of luck and happiness
And please excuse me for my bad English cause Im Asian