Of course you are only trying to do what's best for yourself and the children. I hope you didn't think I was implying otherwise - I just want to make sure that you aren't minimizing any threats to you and get some local advice.
So, I would suggest getting the number for your local police station and ask them their thoughts.
And call 1-800-799-SAFE (7233) which is the National Domestic Violence Hotline and they can point you in the right direction.
It's a difficult balancing act you are trying to manage here and the more RL support you can get, the better, imho. Because your safety and that of your kids is NO 1, yes?