I actually found guilt a very powerful source of motivation for me to get sober, and then to work on my recovery. I just could not live anymore with the kind of guilt I had towards the end of my drinking and it really helped kick off a change. It's not always easy because I still fall into some old behavioral patterns at times (without drinking), but the big difference is that I am now able to put an end to it right away instead of continuing it in a compulsive way. I really feel that I'm able to use guilt as a constructive force in my life, and as long as I don't stay stuck in dysfunctional behaviors, it never becomes crippling anymore, more like a momentary reminder that it's time to watch what I do.
I also did/do what Gottalife said, talk about the past things and what's still not kosher in my present... many people use the 12-step program for this, I use therapy and sometimes trusted friends for specific issues. I also like to understand what leads me to behaviors I feel guilty about (I've found it's a lot more than just drinking), this helps me change the patterns. Then act differently. I actually think that guilt is not bad if it's not extreme, it shows that we have a sense of responsibility.