Well, the older me....would be on the phone with my husband - flipping out about his mother and tell him to straighten it out now or I would. Then I would anyway! lol
The newer me...would be angry and upset. I would wait until I was calmer to make a decision. I would calmly talk to her about what I was told, hear her side and if I felt that she would continue to be toxic to my children - I would either supervise the visits or stop them entirely.
Your kids may love them, but loving toxic people is how most of us got here to begin with.
P.S. I am curious how your ex was ever homeless and why they didn't take him in???