Do you have a PLAN, if he shows up at your door and gets out of control?
(A plan is a discreet set of actions and things you may need - someone on call speed dial programmed into your phone in case you need, an alternate place to stay, an extra set of keys and cash - not just thoughts about it...)
And does he KNOW you have a restraining order?
CLMI