Big Hugs jj
You are at an important juncture. You know what your boundaries are for you to live the best life for you. Hold strong to those boundaries. You deserve a life of peace where your wishes are heard and respected. His feelings are his problem, you have no control over what he feels or why he feels it. Don't accept the blame for his "stuff".
Sending you strength and calm while you deal with this.