Oh man, I can relate to the anxiety about withdrawals. Obviously the feelings of anxiety don't help the situation!
Try not to freak yourself out. The internet medical stuff is enough to scare anyone. What I'd recommend is talking to your doctor. My doctor was cool and just said that I'd be okay to just stop but would feel rough for a while. I also got referred to a therapist which was good.
Once you're through it, it's done and you don't need to go through withdrawals again :-)