Sometimes it can even make you look even more intelligent because instead of blurting out an answer to every question, staying quiet for a while proves that you take time to think about the questions and look at all the sides before you answer, as opposed to just reacting.
That's true.
When I was taking classes, I'd usually spend the entire semester almost completely silent. I rarely answered questions and interacting with other students was almost out of the question due to my phobias.
Anyway, one day near the end of the last grading period, a group of students were stressing-out about the impending exams and talking about test time, when one of them came and sat beside me and said "If I have to cheat, it's gonna be off this guy, he's the smartest dude in the room."
A couple of them nodded and agreed, and I just kind of smiled and played it off. In-spite of having never talked to any of them, most seemed to have come to the conclusion that I was a brain.
It was a funny effect of my coping. I was just being quiet because I was scared to death, and I just listened really intently to block-out the fact that I was scared to death. Ha Ha