I saw the heading and thought I’d post because I have experience with a cheating spouse...but this is so much more. So much manipulation and betrayal. I don’t know what I would do but in my non expert way I would probably look at what it was doing to the kids in addition to other factors. Seems like it would be pretty tough on them as well as you to have him kind of in, kind of out. I agree with others if he’s making real and positive changes to the point you may be able to forgive him then maybe. But can you talk to anyone, like a professional? It seems like you’ve been a safe haven for his bad behavior and may need another person to help you maintain some boundaries.
Overall it just sounds like you and your kids have endured so much pain. I’m so very sorry, and I hope you are on a path to better treatment—from him, from yourself, etc.