Your reaction to him isn't voluntary... a version of PTSD likely kicks in whether you want it to or not. And that's real. And frankly, it IS debilitating at times. So you might be correct that you can't do it yet. If you know that deep in your heart, that's okay for you.
Ask yourself--if he'll be "charming public guy", and you don't look at him or talk to him, and you have a strong friend buffer, can you do it? Or will it ruin the day?
Secondly...will your daughter have similar reactions? (If the answer to that is yes...then I'd change the venue or reschedule for your house another time, because it would be a bad experience for her. Ultimately this party is about her.)
If she'll be fine and you won't, you can still change it if you want (you're doing the work) or decide not to go. Not going doesn't have to mean he "wins". He would "win" only if you stay home, show him he got to you, and remain PTSD all weekend. If you stay home and keep your inner calm (flippantly: I don't want to be around you, so have at it, here's the list of things you need to do to pull it off, daughter is pleased you'll be handling it (this part so he has to own it)), he still doesn't win. You keeping sane and not allowing him to crash your serenity is YOU winning. And he'll know it.
You're in charge of you. You get to choose your sanity. Oh, did I mention it's so much easier to write this than to DO it? So please bear with me.
Question for you...if he weren't going but you came down with a horrendous, honestly can't get off the floor stomach flu with chills, fever, etc the day before the party and knew it wouldn't go away in time...what would you do about the party? Remember, you are physically incapable of going (none of this codie working through the pain stuff). I'm curious, because I think this may be the same thing. It's just that you don't feel like you can give your current "symptoms" the same weight.
This really isn't a matter of toughing it out. Take care of you in whatever way you need.
Too bad I don't actually have that bodybuilder or I'd send him over.
Hugs, and prayers for your peace. Don't forget to eat even if you feel sick. I'd tell you to sleep too, but then you'd probably just laugh at me. So eat something yummy...and if it's sunny where you are today... step outside and just breathe for a few minutes. Just breathe. You'll be okay.