I understand that the police considered him the victim and you the abuser in the latest incident.
It still doesn't make sense for a court to order that the two of you ever be together in the same house with the children.
The court obviously believes that there is a lot of back-and-forth animosity. At a minimum, your lawyer should ensure that the children don't have to witness whatever ensues when the two of you are together.