Jessiec-
I found situations like this hard.
They made me feel crazy, because I was the only one getting upset and "rocking the boat."
Often too I was dealing with my ex being in a black out situation with his drinking so he was not capable of remembering how bad it was (that is not an excuse, just my experience).
Your experience was crazy making, not you. For my relationship this type of behavior was just part of the denial he did not want to change it and I did not either for a long time.