I'll reiterate what Dee said about standing by your boundaries being the right thing to do for yourself, and even go a step farther to say it is also the right thing to do for him. Every human has a right to the consequences of their actions. It's the only way we learn and grow.