Your situation sounds so similar to mine and can totally relate to you on this ... I too knew what I was doing was wrong at the time and I cried a many of times over it. I had a lot of guilt over it. But got to the point where I just put it out of my mind because I didn't want to give it up. I felt like I had finally found a little happiness in my life that I hadn't had in a very long time. And just like you I stayed true with my heart for 21 years. And also I know this would of never happen with this man if it weren't for my A and being so unhappy with my life.
Mine knows also some about this man too as well. And has only caused me more problems in my marriage. But I take full responsibility for what I have done. And he knows as well he is partly responsible for this also. But has he taken steps to try to change .. no he hasn't. And I know he never will even if he did quit drinking I think it may be to late anyways. So all I can do is try to find peace and happiness within myself. Because I know no one can do that for me except myself.
It really sounds like maybe to much damage has been done to you even though he has quit drinking for you to be happy. So maybe it is time to let go and try to find that happiness that you deserve... (I know I am not just the perfect person to give advice here being I am still with mine) ... Hopefully though your counselor can help you through with all this and you can finally do what you think is best for you.
I really wish you all the best with all this. *hugs*