IMO, you're being far too patient with this loser. I'm concerned about the state he'll be in once he wakes up (i.e. dangerous or violent towards *you* and not just himself). Can you have someone i.e. his brother, come show him the door?
Don't worry about "caving". Worry about getting the moron out of your house.