My personal opinion, there's a lot of things going on and I don't know the whole story, so I can't tell you what to do, although I can see both sides of this.
I would say, if your husband feels he cannot miss the meeting, or go to a different meeting, would it be possible to take the whole day off instead and do something during the daylight? You said he's turning four, so there's no issue of him skipping school or anything, and to be honest if my parents had taken an entire day off for my birthday when I was little I would have felt very special.
Again, whatever you decide, it's your decision - I just thought I'd pop in with an "outside of the box" thought just in case you needed one.
And Happy Birthday to your son. <3