Givenup…..for what it is worth...you say that his opinion of you doesn't really matter...
From everything that you say, it looks to me like his opinion of you matters very, very much.....
I think that is why you get so very angry when he storms away or stonewalls you.....It might be "natural" to feel anger in that situation....but, it is lack of self awareness to say that you don't really care...(denial)….
You care...I am willing to bet my children's milk money on that...lol...