I think that if they care about how many friends you have then they are not worth being a friend imo. I always hated those friends growing up who would count how many friends I had and I felt like I always had to compete with them to make friends.
I would say I have 2 friends who I have been friends with since Highschool, and a couple others I met in college. They are guys I can call up any time mostly, if they are in town, and hang out. I recently just dropped a friend that I had been friends with for a very long time, since like middle school. We used to be best friends but things changed a lot and it seemed like we couldn't let go for some reason and I just decided to finally end it.
But yeah I know what you are saying about making new friends while not having many or any friends to begin with. It is actually like that now in my life because I have actually made a couple of friends at my work, one guy I work with and another who works out there and it seems we have a lot in common. He wanted to start doing some activities there together but I started thinking "We will hang out a few times, he will realize I really don't have many friends, and then he won't want to be around me". So I have been sort of reluctant to get the friendship more into motion. Sometimes it is hard but if you just admit it to them that you are sort of a loner and that is just the way you are, it sort of empowers you and they can see you really mean it and they will respect you for it.
Friends have been a tough thing throughout my life. It seems like I have never really had a hard time making friends, but keeping them is another story. I always had things in common with other kids growing up like playing sports and video games and interested in girls and all that so I sort of fit in but for some reason most of my friendships didn't go very far. Not to mention I lost one of my best friends in highschool after a fight we had and our ego's were too high to just be friends again. I had been friends with him since elementary school so it kind of sucked, and along with losing him I lost two other friends with him who were kind of my "crew" guys and this really hurt. I also made this really good friend freshman year in college, I mean we did everything together. We were like really really close buds, even joking about how we would be each others best man in our weddings and then for some reason things changed and we started hanging out less and I don't even talk to him at all now a couple of years later. That one really hurt a lot since I put so much time and effort into the friendship and he just sort of ended it. The friendship started out great but he started realizing I didn't have any friends and some comments hurt my self esteem and I started thinking he respected me less and less since I didn't have many friends. He had a ton of friends and was really well liked but yet he seemed to want to be around me, until I sort of did some things to his other friends and I think he had to choose between those guys he knew for a long time or me, he chose them.
Friends are tough, you want them because they are fun to be around but putting time and effort into them and not knowing what the outcome may be is the biggest struggle for me. I just don't want to be hurt by friends anymore, I will leave that to girls.