I am so sorry! I want to say how much I respect that even though you knew something was up you did not go into codependent behavior and throw fits and demand to know, etc. You let it come up in its own time and you threw him out! Good for you!
I have gotten some slack over this...but it is what it is. I believe that even when someone is in rehab and especially right out, or working a program or whatever...they should not be in a committed relationship for a long time...as in a year or more...They need the time to work on them. And you need to work on you. I recommend a Celebrate Recovery or some sort of meetings for you to get support and work on your feelings. The thing is, right now he is trying to win you back. So he has motivation to do all these great things and work a program,etc.
The only way someone can get themselves clean is to truly want it for themselves. What happens when he does have you back and he no longer has to work for it? Many times, back to the same old behavior. That is a tough drug to kick. He is going to have to put in serious work for it.
Good for you for taking action. Now work on you and let him work on him. If you allow it, God will help you both.
Good Luck! Keep posting, you are never alone in all of this!