Liss, I have never been one for hanging on... so I would say you need to cut all ties except for arranging foe him to see the kids if he even wants to. You need to focus on your self you do not need anyone in your way. He sounds to me like a selfish leech.
You need to file for child support and custody and have it all done through the courts and next time he shows up to mooch the food that you barely have money to buy kick is ass to the curb... tell him to go put it on his tab at the bar.
He has money to play and you can't even eat or get around WTF
I know I can be brutle...