No is a complete sentence. You can say no and he can guess why and you can change the subject and not say. He knows they are not legal in that environment.
Don't be projecting the future.
I do recall a few times I REALLY wanted to be comforted by my xhusband or my xabf.. and every time they just could not come thru. I still wish for that.. but they are both gone and it ain't happening.. but it wasn't happening when they were around so it really does not matter.