Where is their mother in all this? Do you have a good relationship with her? Perhaps she can explain to them that their dad is not doing well and that it would be best if they didn't visit this summer. Either that, or if you want to, you can have them stay with you and allow your son to visit as he can while they are there, but not allow him to stay in your home overnight.