I have considered that too. And I have done just that, and I get over it, and life goes on, but then something else will happen, and I get that familiar, but not so much as before, pit in the stomach, and I know that for a time, I become less loving, and less warm to him, and sometimes I think if he'd only ask, I'd tell him why.
But he doesn't, and I guess that's part of who he is. He likes to avoid any unpleasantness or conflict, where I am happy to dive right in, because I think it makes for healthier, more authentic relationships.