Take them. The doctor knows the facts about them, plus he/she is well-acquainted with your symptoms. The great likelihood is that they'll calm your anxiety just as they're supposed to do.
If not? There are many other medicines to try.
It is highly unlikely that NOTHING of any sort would help.