Personally, that sounds like marriage counseling territory. I know you are supposed to let the past stay in the past, digging up old wounds isn't healthy. Part of that is being aware of what you're saying and more importantly, why you're saying it. Are you flinging mud because he hurt your feelings *now*? Then talk about how you feel now.
My husband brings up the past to build a case of why I'm a bad guy. It's irrelevant to the current situation though. The past is the past. That doesn't make it hurt less but if I can't get over my hurt then I need to get over my husband. I can't have both.