Why not let your lawyer send him a letter, simply stating that this is to inform you that your visa is no longer in effect and if you wish to remain in the country you should make other arrangements with immigration authorities? And that if he has any questions, he should contact your lawyer, not you.
All lawyer has to say, if he asks, is that you are obligated to be truthful with immigration concerning his status, and you simply told the truth. Nothing else needs to be said. For all he knows, they contacted you to find out.