Benzo withdrawals is very, very subjective. Many folks I have read about on that dose find the first few weeks difficult. Others can quit with some mild annoyance.
Here's the good news from my limited perspective: You're doing great. I would be careful about reading the horror stories of others (like mine!) getting off benzos. For the most part, they (me) were on a far larger dose and for a much longer period.
You are going to have periods of anxiety, maybe even high anxiety, and inner-ear and balance problems are very, very common.
I think your doctor is spot on in saying you will be fine in a few weeks. And other than reinstating a benzo, there isn't really much to do other than to ride it out.
You really should pamper yourself during all of this. The walks, resting, just chilling as much as you can is the best advice I can give.
And I'm glad someone is with you to help.
When we get off a benzo, the brain is basically readjusting over a period of weeks. It takes time for our neurotransmitters to fire right without the drug.
Thanks for sharing.