Ummmm, whatever he was doing was probably not a great thing for a married man to be doing. You wrote yesterday that you were both pretty much done, right?
I would use it as a catalyst to get yourself out of there or get him out. No debate or details needed. Won't benefit you now, I know firsthand.
Hugs and god speed w the job.