Hello, I struggled with the divorce concept for a long time also. Just because I get divorced it doesn't mean I'm unfaithful to him.
In marriages and relationships people need to know where your bottom line is. That's hard to do if your afraid to set down your foot and make priorities. I started off by being upfront to myself on how I wanted my life to look and then acting on that plan. Action sometimes starts very small.
There are priorities which he has forgotten about that he needs to straighten out. Sometimes the best way to do that is seperating. If he gets his act together you can always get back together in a year or two. It doesn't mean you have to divorce only you can decide what that next action will be.