I manage my time off work and take him into consideration a great deal, he does not do well with time on his hands.
I know it seems helpful to keep him occupied and not let him have too much time on his own to deal with, that simply REINFORCES the fact to you that HE cannot be trusted. trust is built thru responsible reliable repeated consistent actions on the part of the other....they DEMONSTRATE that they are trustworthy thru their actions and deeds. but if you are controlling his environment and not allowing him to be who he is and do what he does you will have nothing to watch and witness.
think of it like training wheels. we'll never know if we can really ride a bike until we take them off. it' scary, because those training wheels are our security blanket, they keep us from falling over. but they also hold us back. taking them off DOES bring risk....risk of pain and possible injury.
you won't know if he IS someone you can trust again until you let him operate under his own power.