Humans can always construct way worse scenarios in their heads due to fear, than what could ever possibly happen.
I went through the whole "I won't find anyone else who loves me" fear with my exabf. Well, how on earth could I even say that he TRULY loves me? He can't love himself, and he sure seems to love his "mind eraser" cocktails a lot more.