Unintentional hypocrisy

Most people don't put a lot of time and effort into studying their thoughts and actions, especially whether they're consistent.

Which means that if you worry about others' opinions and judgment of you, you'll most likely notice that other people are doing and saying things that sometimes include silliness. If they were as sensitive and thoughtful as you are, they might have reason to be concerned about what they said. 

So while they're judging you for something not 100% "appropriate" - or maybe you just think they are - these same people are "guilty" of doing the things they judge others about. They just believe that their speech and actions are always appropriate. 

This is a slightly convulated way of saying that because they don't doubt themselves, they don't examine their actions as closely as you do, and as a result they feel free to do what they want.

Maybe you can have the best of both worlds. Continue being self-reflective while also being
confident. We're all entitled to some silliness and we all have foot-in-mouth moments. Just be sure to notice your other moments as well.