What other choice is there? Ignoring the elephant in the room? Walking around on eggshells, hoping it will go away by itself? I think all you can do is make clear and honest statements about what you see and what your boundaries are. Then he'll do what he's going to do and you will do what you need to do. It does reach a point where everything has been said and then all that's left to do is take care of yourself.