Welcome to the wonderful world of being a codependant. No one deserves that kind of BS. Unfortunatly the game he is playing with you, they have ALL played. It's called manipulation and trying to make you the crazy one. So far I see that he's succeeding. You need to detach and get your life together. He should not be your responsibility. He should not take up all your thoughts. He should not be giving you negative thoughts.
You need to start taking care of YOU! Thoughts of what you want in life, dreams, asperations, goals. Suddenly you will see the relationship for what it really is, and then you are no longer the crazy one... he is. What you do with the two of you after that is up to you.. but it gave me the courage to leave and take a big breather from AH.
The ONLY thing you should be thinking about IS YOU!!! Remember that.