Don't beat yourself up or give a second thought to what he said about you. Someone once told me that when you point a finger there are three pointing back at you. It's easier to try to hurt you than deal with his own stuff. I also think this is him trying to continue the game.
My xabf would stretch out contact with me when I did something to upset him. I would begin to heal and adjust to the fact that he was gone, then there he would be again telling me he loved me and wanted to be with me. It was very hard to do, but I learned that I didn't have to respond and I also learned that I was worth far more than he would ever give me. He emailed a month ago. Instead of freaking out, I just looked at the email, decided nothing good could come from reading it, then moved on with my day. You'll get there too! One foot in front of the other.
I wrote something about when they leave you in a previous post after discussing this topic with my therapist. I don't know how to link, so I'll bump it up for you! Hang in there!!! I know it's hard, but keep breathing. It gets so much better. I never thought it would either and my life is so much better without xabf.
Hugs!