Sometimes we just need to take a leap of faith. Think about it--worst case scenario. Contested divorce. What would be the worst that could happen? What do you feel you could NOT walk away from? You have no kids together, and he's going to ruin your finances if you stay. Lots of us have walked away from stuff we had a "right" to, legally, for the sake of getting it done.
If your credit rating takes a ding, you can spruce it back up in short order. Sticking around isn't going to protect it.
When I left, I went to a motel for a week to do some planning in peace. Then I stayed with my brother for a week while I found an apartment. Everything fell into place. No disasters.