From what you've explained it sounds like you're allowing him too much space in your head and too much control over you in that you'll be there when he wants you and not when he doesn't...it's quite dangerous to just hand over the reigns to your life and tell them "when you're ready".
It's less about he wants and more about what YOU want and need. Less HIM, MORE you.
If he's confusing you and keeping you dangling then you have the power to stop it and put some boundaries into place to protect your feelings and well being.