It sounds like you are getting more used to the idea of letting go when you said "I think I should give all this up and be done" you are getting there...it just takes time. The only person you can change is you. Envision yourself in your own place living a peaceful life without this toxic person bringing you down. You can create a life for yourself that you are happy and content with. Making that statement could be the first step toward that!