Dear Anonk, I sure know the feeling. When those we have been there for won't or can't be there for us in our hour of need---it feels like being kicked in the gut by a horse!
Sometimes, it takes a painful experience like this to "educate" us to the fact that he is not going to be what you need him to be--as sucky as this is.
So--he needs unfettered space for himself, alone, now--o.k. fine--understood--I think you are trying real hard to give him what he asked. o.k. BUT, THIS IS WHAT I WANT TO SAY TO YOU---He is taking what he needs right, now. I hope that you are not just going to sit nicely, on a shelf, waiting for the time when HE decides that he wants to check back in!!
This is time for you to take what you need. To put yourself first. He may never come back...or maybe, he will... The point is--you do not owe him anything further...
sincerely, dandylion