"He said I didn't trust him" --And there's no reason why you should.
He just wants to continue with the status quo--you all living together, and him continuing to drink. That's what it sounds like from your post.
Your son deserves better, and so do you.