I'm sorry you're hurting, zjw. Nothing is unforgivable. As one or two others mentioned, have you thought about counseling? For you, for both of you? A good counselor could provide you with the tools and space to dig gently into some of these issues and begin healing them.
You're worth it.