That's true, in fact you have probably already become an unreliable source. He's smart, he knows he can be tired and you can be tired but you don't forget conversations 5 minutes later.
His friends at school were probably all tired today from heavy duty Halloween collecting and perhaps getting to stay up later, but they all remember what they just said.
Being tired and forgetting what you said 5 minutes ago is an exception (we may all have done it from time to time) it's not a norm. So not only are you lying to him, you are challenging his perception, that can't be good?
The reason I suggested also telling your AW is not because I think that she is going to do anything in particular with that information but she might on some level remember it when drunk and that would be helpful perhaps.