Can't say that I feel quite the same, but it's definitely not pathetic. :3
I love video games because I know that the characters and monster are anything but real, and so aren't capable of judgment. Just 1's and 0's aligned to create a visual reference that represents a person/creature. Online games however, provoke allot of anxiety in me, even though they can't really read my body language.
But as far as offline games are concerned, I have no anxiety at all. Heck, I even love the scary ones like BioShock and Dead Space. They make me feel safe almost, and yes I know how ironic that is. x3 It probably has to do with the fact that I feel safer in the dark..
Anyway, you shouldn't feel pathetic. Those characters/monsters are designed to look real/appealing on a social level, it's only normal that they initiate a feeling of discomfort amongst those whom have social phobia. Especially Ocarina of Time and Majora's Mask, it might be colorful and fantasy, but there are parts in both games that even made my skin crawl.
Shadow Temple, Ganon's castle, Ikana Valley, The fish boss thingy.. to name a few.