You didn't do one thing wrong, you were responsible and considerate.
He was not. I call that control and verbal abuse.
I won't listen to it. I am not in that situation with hubby...but when someone starts disrespecting me, I pick up my purse, keys and such and turn right around and walk out the door. go to a motel if I have to...or back to spend the night with friends or family. And I don't contact them to tell them where I am and I don't tell them where I am going. (From other prior relationships)
Don't let him win this and isolate you. Go out MORE often.