As has been mentioned the point of leaving is the most vulnerable time for the departing member. It is essential that you arm your self with a safety plan. Talk to a worker at the local domestic violence center---do it from a safe place. Also, read-up on one of the many good domestic violence web sites---make sure that you do it from a safe computer.
Just because he hasn't been physically violent, thus far (if he hasn't), does not mean that he won't.
dandylion