i agree with noday. call the police station and tell them what happened.
i am not an expert, but if he would not let you leave, that could be considered kidnapping, holding you prisoner by the threat of violence against you or your baby.
two hours of his hands fisted up, obviously angry (huffing and sneering).
if you really do not know what he did that could be considered criminal, then please call a domestic violence hotline and ask them.
i do understand how it can seem "not that bad".
i have been there.
Beth