As silly as this may sound, it was kind of a revelation to me after I'd been sober a little while that when I stopped doing the things I felt guilty about, I didn't feel guilty anymore.
I also learned that it was actions, not words that counted. It didn't matter how sorry I was, what was I doing to ensure it didn't happen again?