It's difficult to escape that doormat/victim mindset.
Try to not look at it as, "he's doing this to me." Instead take the approach of, "I will gently remind him I'm not interested."
He may get it and he may not. It's not about him. It's about your recovery in the world.