I can relate to that feeling. I remember always having this incredible fear of ending the relationship only to discover that it was a huge mistake and that I'd have to live with the regret for the rest of my life. Unfortunately, I'm not really good at making big decisions and so I tend to not make them if I can avoid it. I remember thinking that if I left though, that my AH would get his act together and become the person that I knew he could be and that some other woman would get the better part of him and I'd be left with nothing.
I also felt those fears that you mentioned as well. Fear is huge!!!!!!