I hear you on having to prioritize what you take now. I had to go (back) on Cymbalta (similar w/d as Effexor, unfortunately) to get off the opiates and am on the lowest therapeutic dose. I credit that, and strenuous exercise, for avoiding much of the panic/anxiety I had once before when I tried to quit opiates.
Hang in there - you can do it!