I wouldn't unilaterally stop visitation, at least not without a lawyer's advice. You'd be risking contempt of court, and depending on the judge, you could get sanctioned. Not every judge would see that the harm is so imminent you would be justified in violating the order without seeking to modify it.
I don't practice family law and couldn't give legal advice in any event. Talk to your lawyer--that's what you are paying him/her for.