A 3 day detox is usually the pause that refreshes, when the drug is heroin. Alcohol/ pot is a common gateway back to the drug of choice.
He may be shutting you out to focus on his recovery. He may also be doing so to use, without the hassle. What people say and do are often two different things. This is most true with people playing the in recovery game.
He's living his life as he sees fit and this includes continuing to live with his alcoholic mother and associating with people who use a gateway substance, minimally red flags.
Give this guy the space he's requested and the dignity to work his recovery or not. Ignore the words and pay attention to actions. And know that you can get off Mr. Toad's Wild Ride anytime you want to do so.