I also think your X may have a good point wanting custody of the child.
if your current husband treats you this way how does he treat your child? if he treats your child good ok great but if the home is crap because of all this how is that affecting the child?
all things to consider.
I grew up in a household with a verbally abusive and phsycially abusive step father. it was crap the verbal was far worse then the physical I think.