Sounds like there are two issues here for you: your limits with the baby's dad/family and the pain he has put you through. You do need to heal, removing him from your life right now is probably the best plan so that you can focus on the baby.
Again, I hope you think about therapy and here is why: regardless of how messed up he is, he is still the dad. He will always be in your child's life. At some point you will need to communicate with him. If he checks out of the child's life, there is nothing you can do but Id caution against preventing him from even trying (assuming he won't ever harm or neglect baby.. then all bets are off. An active addict isn't a good parent so you are wise to keep him away at this point). If at some point he does get well and could provide support for the baby, you don't want to burn that bridge is all I'm saying.
It isn't anything you have to dwell on now obviously. The most important thing is your well being and baby's well being. Just food for thought since I work with families and see stuff like this every day play out.