Sounds to me like you have good, solid values and appreciate the good qualities of your spouse and your life.
There are people in my family that are poison to me. I can't stay around them for very long. I used to feel guilty about this. I have learned that it is not my fault that they are wacko and I don't have to mess myself up by being with them. I love some of them but I don't have to be around them. As I get healthier, I can take a little more of them. There is nothing wrong with staying away to heal. If this takes a long time, its still ok.
Good luck.