He could be charged with false imprisonment. If you felt threatened/thought that he would harm you if you tried to leave--that is still false imprisonment. False imprisonment does not require physical constraint.
I would call the police if you feel threatened. At least the report is there IF anything else may happen. A pattern has developed.
He does not sound mentally stable. Do you have family or friends you can stay with until this is resolved. Some place you and your daughter feel safe?
Do not beat yourself up for having a child with someone who is not mentally sound. We all do things that we regret later. You recognized that he is not someone you should ever be around--that is what is important.
Also, please call a domestic violence agency. Don't minimalize what happened. Just because you were not physically restrained does not mean he did not abuse you and your child. They can help you.
Please, please call the police and domestic violence. They can help you.
Take care of yourself and your daughter. Keep posting and let us know how you are.