Thank you for the very real advice everyone offered. I confronted him this morning about finding his stash in the kids bathroom. He immediately apologized and said that he was very wrong to leave his stuff there and it will not happen again.
If I find it again, I will dispose of it. I didn't tell him that, the conversation was going in a positive direction, so I didn't want to throw in anything that might come off as a threat to him.
He then gave me the, 'I'm gonna quit at the end of the summer anyway' line.
Looking back at the conversation, I can see how he manipulated his way out of getting his butt chewed out by being all sweet and apologetic.
Anyway, the boundary has been set and I feel a little more confidence in being able to set some more and not toleate things that are obviously unacceptable.