This man has told you how he feels about women. Don't think you are enough of an exception to his rule. Secondly if he is an A, then A will come first.
If he wants to keep rehashing this one small miscommunication that you truly apologized for, I would take this experience as hurricane warning. Do you want to deal with a hurricane warning every time life happens?
Just reflect on what you told us, you are being warned by others who know this person too. Honey, you can do better. You need to build yourself up so you believe THAT.