Perhaps a few sessions in relationship counselling might help. He would hear how you are feeling about what happened. It might help you if he were able to talk about the impact on him.
Alternately maybe you could see a counsellor and discuss what happened?