I can tell you how I have handled similar. I will not allow my xah to stay overnight at my house. I will allow him here to visit during the day but he can't stay overnight. I trust that he isn't going to steal me blind or anything like that so I would even leave him here alone for an afternoon, but he can't stay overnight.
I've held that line, even at Christmas. I have an ex that is dying to see his kids too. Yet he calls sporadically, doesn't send them cards or letters, and can't figure out a way to pay for a motel room. He cuts me off when I talk about them so he can moan about himself. What he wants is to move back in here so he can go back to his easy life that I enabled.
If he wants to see his babies badly enough he'll figure out how to do that without staying at your house to do it. He's working, he's not supporting them, he's living with his mother, he says he's not spending all his money on booze, so one would assume he has an extra $200 to get a motel room when he comes down to spend time with them.