If the rehab place offers a 'family day', I'd encourage you to go.
Your worries and fears are common and family day's are there for you to help process what you have gone through with the addict.
It can't cover everything but it may lessen the anxiety you are feeling in addition to Al-anon.