I think for your own sanity and well-being, you should go no contact with this person- and it is going to be the best for him, also.
You can let your guilt go, it's ok. Is your life better? Sounds like it- and what would it have been if you stayed? There's really no way to know but the likelihood is that it would be much worse. Doing what is right for yourself is fully allowed- for some reason we have a hard time with putting ourselves first- even when it means our demise if we don't. Do not let your PAST relationship taint your current one- go forward and live your life, your ex will have to do the same. He may need a hard line- and that means cutting him off. He will try and play you like a fine instrument- don't let him.
Go live and be well. You deserve that.