Being famous is stressful

You encounter too many assholes unless you just don't want to interact with the community. Which then makes you seem like an asshole, it's a great catch 22.

Don't get me wrong, in every day life there are still assholes, but you don't have to worry about them or keep up a "face" because you're a role model for anyone (generally). 

Basically, don't be an asshole if you can.