My husband is staying with his parents (and apparently drinking again). Each situation is different. I pounded him over that one and we both, thanks to advice given here, signed a contract stating we will not drink when the children are in our care or 12 hours prior. Now, I adore his parents. So, I do trust my children there. But, I did call them and DEEPLY expressed my concern and the lengths I'd go to protect them should the agreement get breached.
If it's not a parent, I wouldn't do it at all. If it's a grandparent, I'd hang out and visit also.