Treat him like you would a business client.
If you get a call at work, you take the call if it is convenient.
Same with your AH. Speak to him only when convenient.
If your client calls and wants to discuss gambling tips, you would change the subject (unless you are a professional bookie).
Same with your AH. Stick to childcare, visitation, finances, calmly.
If your client can't reach you by phone, they leave a voice mail.
If it's an emergency, they call your partner/secretary.
If it's important info, they can text or email the information for you to read at your convenience.
Same with your AH. Let him leave a message.
He calls the kids, let them pass on the message. (there is no obligation to return the call immediately)
If he needs to pass on important info, let him text it or email it and you can retrieve it at your convenience.
If a client shows up without an appointment, they only expect a few moments of your time. The client doesn't expect you to stop your day.
Same with AH. Stop making accomodations for his interruptions of your day.