Will he go to joint counseling with you if you want to keep trying? It sounds as if he has anger issues. Name calling is off limits and being told to shut up is just plain RUDE. When he is not angry have you asked him why he does this to you? Or are you afraid if you ask he will become violent?
If you move back to NJ and take your son it is not kidnapping as long as you keep the lines of communication open between him and his son.. talk to a lawyer on this one. If the relationship is over, then it is over and you have the right to leave it.