I think I would call CPS and tell them her history, the police being called, the drug/alcohol situation with the both of them and another child in their care every other weekend. If you are willing to give them your name you can also tell them about him dropping the children with you so that he could find other housing.
God forbid something were to happen to the baby or that little girl and you didn't call, that would be some heavy guilt to live with.
It's not an easy decision but someone with a clear head needs to advocate for those children.