I went through this this Fall with my AH. We weren't separated though. I got to be friends with a group of folks from work this year and we all would go out from time to time... Some men, some women. I told ONE story to him that my friend D told me (man) and ever since I've been accused of having had an affair.
What I think? A) My own AH has had an affair and it's projection or B) my H like yours is freaking out that you might be stepping outside of his control and seeing things more clearly and he's going to try and use anything he can to suck you back in. If you change then he has to either face reality or find a new way to continue to delude himself fast. It's WAAAAAY easier for him to make you think you're crazy or wrong or bad or ______ fill in the blank, then it is for him to face reality.
No one, not even your AH thinks you had an affair. Smoke and mirrors with a healthy dose of quacking is what he's doing.