I agree with Anna. I'd tell them that due to legal and personal considerations, the pool invitation is rescinded until further notice.
Where I live, a pool would be considered an "attractive nuisance" as it could attract people and accidents. I'd tell you that you're afraid of someone having an accident and that you've been advised to make the pool off limits.
I'm sure they'll be upset, but too bad. If they want a pool they can get their own.