The one thing with A's - there's never a "good time" to leave.
With XABF, he was always planning things a year in advance, and paying for all of it as soon as the venue would let him. This means that any time I left him, there'd be upwards of $1,000 worth of hotels, theater tickets, dinners (he liked to buy gift cards in advance, too), and who knows what else floating around, and with what he spent, he'd be lucky if he got half of that money refunded to him. I know he had 12 hotels completely paid in advance, non-refundable, including the Waldorf Astoria in New York, throughout the year.
Well guess what? I had enough. I'm not going. I can't be bribed with expensive trips anymore, because I end up doing all the driving, stuck in some place goodness knows where but too far from home with an empty stomach, and he ends up collapsed on the hotel floor in a drunken stupor screaming obscenities at me.
There is no good time to leave an A.
But the best time is when you're ready.