I think we get caught up in right or wrong black or white thinking sometimes.
Is what he is doing a problem for you? Do you want to live like you are living?
I know how entangled we can get in these types of relationships. Think about you and what you need. What would you be doing for you if your partner was not a factor in your life?