You are very brave and strong (even though you may not feel like it).
As it stands, you recognize that he isn't a person to be married to or father your child. It's difficult because you know what he his potential is from the past. However, that is HIS potential not yours. You have no control over whether he decides to show it again and that will take ongoing sobriety/recovery.
Oh, and they are very good at making you sound like the crazy person! It messes with your head.