My best suggestion would be to sit down with a clear head, and write him an e-mail expressing exactly how you feel, even if you're in the same house at the time. Being able to get your thoughts across in that way is sometimes extremely beneficial for both people in the relationship.