If you've tried talking it over with your boyfriend and he doesn't see the pot as a problem, what else can you do but leave? I am sorry if this is not what you want to hear. It doesn't sound like he is capable of being a good partner, and you can't change his mind for him.
Perhaps telling him you need a break will be a good wake up call, but you won't know until you try. Good luck.