There is absolutely wrong with being angry. Emotions are neither good or bad, they are just emotions. Human beings feel emotions. It's normal.
The trick is doing something healthy with any emotion. Don't beat yourself up for getting angry. That's like getting upset with yourself because you have to poop.
Allow your emotions to happen. Embrace them even.
Have you tried simply telling him when you're angry? Not blaming, just saying how you feel - you don't always have to explain why you feel the way you do.