I kind of felt like that when I was much younger, but I think that was more to do with my age group at that time, as young kids particularly cleave to large groups. Though I can still emphathise with how you feel. But, I have been told before that people automatically assume that I'll be on my own, as I like to be on my own a lot. This being as in a general opinion rather than a negative, as they known that this is my preference. I suppose it would depend on how you are perceived by others, looking anxious and disgruntled may give the impression that you do not want company. Those that know you more may read it differently. But in my experience, people do not normally think less of you, unless they already don't like you, in which case they'll see something bad in everything you say or do regardless of how faultless you may be, lol. I'd recommend you just relaxe, and don't over analyse everything. Easier said and done I know.