First off, sending you a big hug.
Second, I am so, so sorry for what you've been going through, but wow, what a warrior woman you have been on behalf of your children and yourself. It's really impressive.
Don't start second guessing yourself now, especially not based on anything he tells you. You are obviously one intelligent, brave and strong person, so trust yourself. Your gut says get out...get out.
Have another hug.