I would get another divorce attorney STAT, (maybe someone on your side, or a female who specializes in matromonial law)...and simply refuse to pay him....he does not sound that good....he sounds very unprofessional and rude.
get some referrals from friends or people you work with...and he can contact them if he wants to speak to you.
my divorce attorney was almost like my best friend and never spoke rudely to me and either he or his paralegal always returned my phone calls within 48 hours unless it was an emergency....you get what you pay for...he actually sounds very inexpensive.