It doesn't sound like lost serenity to me Tuffgirl, it sounds like you were pushed to the point of responding or rolling over & being a doormat. It really sounds like this joker had it coming & just maybe you'll have given him a lesson in looking outside of himself & his selfish view of the world?
As far as 'what' to do? Nothing! Hold your head high, you've only spoken truths! It's up to him to change or accept or continue on in his ignorance.
I've only ever had 1 issue like this with a neighbor. (I don't count my alcoholic rednecks across the way... we've ALL been into it with them, lol.) I used to do tax work & my neighbor was a client, but a bad one. Always late with paperwork, always disorganized, wrong calculations, expected me to pick up/drop off at her convenience, complained about her fees, etc. The year I quit the company she hadn't bothered to read the notice the company sent out about how I had left but called me at home the day before the W2 filing deadline & wanted me to do all of her 4th quarter payroll reports & get W2's out to employees that day. She kept throwing it back on me like I owed her a solution even though all the fault was on her side & for me it was just the final straw... I'd put up with her crap for years with not an ounce of gratitude & here she was demanding more. I laid into her with both barrels much like you described your own interaction & she has never treated me even close to that way, ever again.