WTBH...I'm so glad you've set up clear boundaries for yourself. At least for now, you have clear cut guidelines that your A needs to respect or he's out.
As for this job...again, my instinct says: a happy woman makes a happy mama and a better parent. I think a lot of mothers tend to put everything kid-related first and end up running themselves ragged and feeling frustrated/unfilfilled. For the first little while at my current job, I was woefully unhappy...it's not at all the job I dreamed of. Having given it some thought, I'm now doing part-time graduate work in education WHILE working this job so that in a few years, I can have access to a field that excites me WHILE being able to provide for DD. It's the best I could come up with considering the circumstances.
IMO, your kids deserve a happy mama. You deserve to be fulfilled in your job, especially after all the cow excrement you've dealt with in the past few years.