I was in a relationship pretty much identical to what you're describing. This was years ago. Things got worse and worse and worse until one day he became furious over (no joke) a potted plant, saying that I was holding him back and I was totally selfish and all kinds of other pretty mean nasty things to me. It was a wake up call for me.
His problems are not your problems. Your answer of "do what you want" is dead on, but it doesn't sound like you really believe that he can and *will* do what he wants. Including baiting you into arguments so that he can blame you for whatever is messed up in his life.
He won't change unless *he* wants to change. The sooner you can accept this, the sooner you can find peace in your own mind and decide what you want for *your* life. When was the last time you thought "I'd really like to do XYZ" where the statement didn't involve him in some way? That's taking care of you.
As for my story - he eventually tried to kill me. I broke off the engagement and told him to go find someplace else to live. He left me in 5 digits of debt. It took a long time to dig out of that hole. The only thing I have to thank him for is that he made me hit bottom and start my process of recovery. I am not the same person now that I was then, and for that I am greatful.