Sometimes people go looking for reasons just because they're really good at the first few weeks of newness and oxytocin highs and rainbows and kittens, but can't handle the realities of negotiating a relationship with an actual human being.
So, in short, he may be full of it and this doesn't mean you have to move to Pluto to avoid the taint of your ex. The right guy will admire what you've been through and survived.
Have a hug.