This has nothing to do with you going out and everything to do with him losing control over you. His terror can be summed up as thus: "What if she starts behaving like I have? What if she starts doing whatever she wants?"
My son calls people like this "douches".
You did the right thing. He's being a jerk. He's also going to try to use it to justify his own future bad behavior, I would guess. It;s very childish.