I used to have a huge fear of spiders. It was almost a phobia. At that same time in my life, I struggled a lot with setting boundaries and I really didn't understand how to do it.
One day he and the kids went out to do something while I was gone. When I got home, there was a big box on the kitchen table. With a tarantula inside. They'd found it and thought it would be neat to bring it home. And inside.
Boundaries became crystal clear.
I wrote a big note and put it on the box:
Dear B: as you well know, I will not tolerate spiders in my home, especially ones that have been intentionally brought in. If the tarantula is still here in the house when I come home, you and the spider will both need to leave and will not be welcomed back. I refuse to live with a spider in my house.
Funny, after that, I started to be able to set more clear boundaries. A boundary is about what I will or will not tolerate, and what I will do about it. It wasn't about him, it was about ME.
Feel free to use the note and substitute some other word for spider.