When they tell you that you are all they have it's a HUGE form of manipulation. If you are all he has it is b/c he has burned his bridges with everyone else. He is in full control to change that, you are not.
Stop isolating yourself. It will set you free and gain you the support you need. Turn the focus back on you.
Tight Hugs.