You probably already know what I think. :P
He told me that I should have kicked him out 2 years ago but that he's grateful to me for not doing so. He insinuated that if I had left, it would have been the end. He didn't specify what 'the end' was, so I can't assume to know.
Is this a red flag? I think so. I think he's reminding you where your place is while throwing just enough cotton candy and rainbows at you to keep you where you are.
The good thing is that you don't have to do anything about this. Just file it with a sticky ("TBD") and keep your side of the street clean.
Many hugs, Liz. This guy is a doozy.