Why do you have to do things always with him? Why do not you go out by yourself? Take the kids out and do something with them? If he is not using maybe he has always been that way, not caring or always getting mad it could just be him.
You are responsible for your own happiness? If you are unhappy what is going to make you happy? Change the way you are because you cannot changer the way your husband is?