Honestly? Yes, I do think you are being unreasonable. It sounds like he's doing pretty much everything right, so maybe you should cut him a little slack. If he starts hanging out again with these guys, you might have a point, but if he's not doing that, then I'd let him decide if and when to have "the conversation" with them.