Can you just split for the weekend? Go see a friend, go camping, anything to get the hill out of Dodge. If you don't want to be there, don't be there.
And yeah, you aren't leaving him at his worst, you are leaving so he doesn't drag you down with him. You have a right to a great life!