He was pushing your boundary, testing you. Good for you for sticking to it. Not feeling well and blaming his roommates for not being able to rest was manipulation, hoping you would feel sorry for him. The minute you bend, even just a little, you're opening the door to being walked on. Like you said, he couldn't have been that bad, since you are the only one he called.
Feeling bad for it, now that I can definitely understand. It's a codie trait that I have as well, feeling responsible for others when they aren't happy or comfortable. It's my job to make everything all better, right? Next thing you know, he hasn't left the couch in a week, my refrigerator is empty, and there's no hot water for my shower... and i'm like "how the hell did this happen? It was just supposed to be one night?"
Well done on keeping those boundaries tight and secure. Once you budge on one of them, the rest are fair game.