You are not wrong by refusing to let him continue to use you!
Your responsibility is to you and your son; he is a grown a$$ adult; with children no less that HE is responsible for; you are not responsible for HIS children... I know that sounds harsh because we ate talking about children but they truly are not your responsibility. And you certainly do not owe him anything.
Cut your losses, get his phone shut off, or at least out of your name and hand the responsibility for his life back where it belongs: to him.
Good luck to you and I am so sorry you are dealing with this!