He knows he can hurt you emotionally and he takes full advantage of that.
I would refuse to speak to him on anything but essential matters, or go as far as text/email only.
Take away his ability to hurt you. Be indifferent and stick to the essentials. You can do this! You will give yourself the gift of detachment. Think of it that way--a gift to yourself, taking care of yourself, caring for yourself, taking your power back, respecting yourself.