So he's blackmailing you?! Seriously, this dude is crazy. My ex threatened me multiple times about me leaving him. Many times. I was terrified, too. Not anymore bc he's just a washed up drunk spinning his lies and threats to make himself feel better I guess? Not really sure. My ex screamed at me one night, "you're not taking my ******* kids, bitch". Yep-his kids are possessions to him-nothing else. And the father of the year has only abused his kids and lost all visitation rights to them a year ago due to drinking. I still have a lot to learn but I do know if someone has to THREATEN you to stay with them, um, it's time to run away.
Btw, most if the time if you are on the right path and walking ahead and telling the truth, the crazies come out to threaten you. DO not let him sideline you, as you and your kids deserve to get out of this hell.