First of all, good for you for speaking about the sexual abuse from your uncle and refusing to be in the same room with him. That's powerful!
And, you're doing the right thing to allow your son to have them visit for his birthday. Beyond that, you need not say anything until you are ready. Stay strong.