Good for you to set the boundary. Now have a plan to enforce it, which can be harder to do than actually setting the boundary. My RAH tested the boundary the next day after I set it. It also helped me to let a few people I am close to know that I had set my boundary and know I could reach out to them no matter what time of day or night if I had to leave. In my case my AH left instead of the kids and I leaving when he crossed the boundary.
My DD is 7, but I had filled her teacher in on what was happening at home in case she brought it up. Something to consider even if your AH comes back after the weekend and doesn't test the boundry you set.