The first time I broke up with my XABF, we left the door open for "maybe in the future we can communicate at least as friends" type stuff. I ended up feeling bad for his pathetic facebook statuses and ended NC when I saw him a few days later at our local Starbucks. I ended up hugging him, which turned into kissing him which turned into taking him back. It felt great for a little while -- like during the hug and the make-up kiss.
Then it all went to hell again, literally it only took him 48 hours. Once he had my ear he went on and on about how our few days apart were the worst time of his life and how obviously that was all my fault. He would come over to bring me dinner and then out of nowhere launch into a tirade about how much pain I'd put him through when I abandoned him. He'd make vague references to bars he had "found himself in" and then tell me that his sponsor was requiring him to go to 90 meetings in 90 days. Since he was very busy at the time, this whole 90 meeting things was ruining his life, and of course by the transitive property...me breaking up with him had ruined his life.
He came back to my life full of even more rage and hatred. Any time he freaked out at me, he declared it was my fault because I'd damaged him when I broke up with him. He remained 100% convinced that I slept with someone, anyone, everyone in our few days apart. I didn't. He punished me as if I had. The post-breakup-back-together period was the absolute worst time in the already ugly history of the relationship.
That's just my story, and talking to your XABF obviously doesn't necessarily lead down the same dark road. But it sure sounds like he put you through hell and doesn't deserve any chances to wiggle his way back in and mess with you again. I am so sorry seeing him at church brought you down. You deserve freedom and peace and a loving respectful relationship. You are doing so great so far -- still rooting for you over here!!