It is ultimately your choice but if I were you I would go on vacation. You can always come home if you need to right? Chances are everything will be fine and your baby will be fine with your in-laws. There was a reason your travel plans weren't canceled!