Keep reminding yourself that HIS recovery is not dependent on what you do or say or don't do or don't say. HIS recovery is dependent on what HE says and what he does. I agree, you do what feels comfortable for you.
Also keep in mind that those counselors are HIS counselors and they will advocate for him and if he begins to fear you may leave him, they may try and get you to agree not to, at least for a while. And they will try and talk you into coming to a "family" weekend or try and include you via the phone in his recovery. One rehab my ex was in they did that and I really didn't know any better and of course wanted to help and support his recovery. I must say that I felt put on the spot and felt guilty if I did not. BUT I learned so when the next rehab came along and his fear of me leaving him turned into his next obsession, they reached out to me and attempted to bring me into HIS recovery and I said NO, that I was working on my own recovery and it was best that he work on his own.