Check, check, and check.
Here's my attempt to partially explain it: I'll start with rowdy boys and authority figures because they have something in common, IMO. I think what I'm afraid of is unpredictable behavior. I'm afraid that they will do something to me (or my stuff) because I was unable, because of my social inadequacy, to defuse the situation. Of course, there's a different balance to each: Rowdy boys are more unpredictable, but have less control over you. Authority figures are less unpredictable, but have more control over you.
As far as members of the opposite sex go, I think it's because of the hyper-critical nature of the situation in the first place. I mean, when you're "single and looking", you're essentially trying to sell yourself. You've put yourself on the market, and thus, have subjected yourself to a lot of judgment and criticism; you're under the magnifying glass, so to speak. People are looking at you and judging; "are you good enough for me." Ironically, I'm married and still break into a cold sweat when I talk to a pretty girl, but I think that has more to do with "How am I supposed to behave in this situation being a married man."
