Sounds like you're playing the "push/pull" game with him.
Your best bet is to keep the focus on you and work on yourself and your self worth.
He's manipulting you because he knows you will take him back
and P.S, canadian winters are no worse than winters in the UK ;-)