you seem to be forgetting one critical person in the matrix......HIM.
assuming you didn't leave him chained to the radiator with no food or water in reach, and assuming that he is not FIVE....he is a full grown adult perfectly capable of taking care of himself and his issues. he doesn't need a mommy, or a caretaker, or someone to whom he can ABDICATE responsibility for his own life.
let's look at this worry of yours....has it fixed anything? has your worry made a difference to his stuff?
no.
but it has given you someone, some THING to focus on besides what is most important...YOU.