Sounds like your husband doesn't respect your wishes or think that your feelings matter. Is this the only area where he does that, or is he showing you disrespect in other areas as well? Are you happy in every other regard? Can you communicate with each other at all, or does he just blow you off? If this cr@p is his way of acting out his other frustrations with life, then maybe counseling would be in order.
Do they really think you're serious about "no more frat parties at my house" or do they think you're just complaining? If they still at the point where they think can verbally abuse you into joining in, then maybe they aren't taking you seriously yet.
It may be time to set some boundaries about what you're willing to take from here on out, and what the repercussions will be if those boundaries are crossed. You deserve to be heard.