My guess is a man will be less likely to start crying if there is another man around.
As for your feelings of guilt: Stop it. He is a grown man, responsible for his own behavior and THESE are the consequences of his behavior. If talking to him makes you feel guilty stop talking to him. When you go to his house, take a 3x5 card with you to hand to him. On that card, write down the number for A.A. and the number of the local Salvation Army. When he starts his boo-hoo'ing and blameshifting hand him the card and walk away. Don't have a conversation with him about it.
It took me a good two weeks to stomp out the guilt the last time I had to leave an alcoholic behind. Every time the guilt would come up, I would talk to myself OUT LOUD, reminding myself all the reasons why this is HIS responsibility, HIS life, and HIS choices.
Good for you for choosing life. (((hugs))) You can do this. Feal the fear and do it anyway.