What bothers me the most is that you aren't secure enough in your own judgement to not only do what you feel is necessary but to also accept the consequences of what you do.
In other words, your worrying about what everyone else thinks of you says you're not real sure yourself.