Maybe you are too understanding and compromising right now.
Have you had your locks changed yet?
I would tell him that he needs to come on Thursday like planned other wise you will pack his stuff up and leave it at the curb on Friday.
You be in control over this, not him.