"While he isn't physically abusive, he put his hands on my throat one time and has done things like punch me in the arm a few times."
I'd hate to know what you define as physically abusive if you don't consider what he's doing to be such. The name calling, punching walls, etc. is all abuse, and as you're sadly having to learn, it's ALL hurtful.
Thankfully children aren't involved...
The next time he gets "like that," I would call law enforcement. This will put him on notice that his behavior, going forward, will have consequences.
Good luck to you.