he has every right to date, you have every right to not to want to have to deal with that in your home.
it's not about right and wrong, it's about living our lives how we want and allowing others to do the same.
You feel how you feel, and that is entirely appropriate and fine, he feels how he feels, ditto.
you don't need validation here. If you don't want to live with him anymore, that is the end of the story. We don't have to have the high moral ground for our feelings to be okay and legitimate, they just are what they are, and they will change