You said "I'm not a tit for tat type of person, but at the same time its like if he doesnt care about me and my feelings, why should I care about his? If he does what exactly he wants to do without thinking about how his actions affect me, why should I care about his.."
I think the answer to this question is you shouldn't really care about his feelings, or how your actions affect him. Not that you should act like a selfish abusive alcoholic, but you need to start taking care of YOU. If he doesn't like it, tough. Detachment is not expecting him to care about your feelings or change in any way.