I'm in a similar situation. I know I have to learn to focus on myself and less focus on him and make boundaries for him. I'm not yet ready to tell him to leave but I'll know when it's time.
I don't have much advice, but justed wanted you to know that other people are feeling the same way you are.
HUGS