He can crow all he wants about wanting joint custody, I see someone who is manipulating for financial support. I think he is pushing your buttons hoping you will 1) cave and let him stay or 2) receive child support (drinking money) 3) Is using the custody as leverage for a better settlement.
If I were you I'd give him one chance - "You can either sign this and have the money I have put aside so you can move on with life, OR you can not sign it, we can fight it out in Court, and there won't be one penny left after I pay the attorney, you will get nothing. YOUR CHOICE".