I don't got too much experience on situations like that, but the few times it's happened... Can I say both fear and shyness? I'm always shy when talking to strangers, whether they're interested or not. But when/if I realize they want anything more than just small talk, that's when I freak out and wanna get away from the situation.
Maybe it's just because the few times it has happened it's been some creepy drunk guy, usually twice my age... But I'm anyways pretty anxious and fearful of "romantic" situations like that.