i don't know if i fully understand your concern or question, but if it has to do with you being uncomfortable with leaving or thinking to leave, let me say, that leaving will always be your choice and only when you are ready.
maybe its time for you to decide what is best for you and try not to base your decision on what you think that others think you should do. do what you know to do to make your own life better. keeping you and yours in my prayers