Yep, I've also heard that numerous times. The worst minimization of someone else's feelings.
The next time I hear it, I'd really like to tell him to get over himself, that he is not the be-all, end-all, center of the universe. Not sure how well that'd go over! (lol)