I used to ask myself the same question, i'm glad you raise it. It illuminates a subtle background theme within our inner world which unknowingly dominates much of our energy. Like some background music or story that goes on and on in constant repeat.
I figured, i had a choice to make. Either i continued to live in my own 'Social Phobia World' and mould myself around what i believed others thought about me or i could start to live in a world free of what i reckoned people thought about me.
So the choice was easy, all it took was answering the question what do others think of me? The answer is simple:
Everything and nothing.
We are constantly changing, how we look and act is constantly changing, how others see us is constantly changing, how others see us is in 'their own world'. We are everything to everyone, to some we are strangers, to others friends, to others a loved one, to a few we are the enemy. We are not a fixed entity of opinion but others will view us that way. What others think of us is transitory also. What others think of us changes in time, we are friends, then we have arguments and so on.
The point is this:
It does not matter what others think of us, at all. Of course we need to show example to our children in this context and have responsibility but should not be governed by the opinion of others. We should be free to act and not worry about the illusion of how we are perceived.
James