Trust issues. Yep, I've been there. Am now.
Have you been to marriage counseling? I don't know if that is the answer, but I would recommend it either way you go in this.
I too share your affinity for pot. Sometimes I wonder if my use is a way for me to cope with a stressful/unhappy marriage, or if that is overthinking it. Either way, I have gone without it for sometime, hoping that would clear my head enough for me to zero in on what I exactly I want in this relationship. And now I'm even more confused than ever.
I can't imagine personally coming back from unfaithfulness, but I do have good friends who have. And they have the best marriage now of anyone I know.
Reiterating, you should probably talk about these issues with a professional if you haven't already. Also, I recommend Al-Anon, if you haven't gone already.