Oh man, your post sounds just like me at 22. I managed to quit the hard drugs around then but kept pot for the next 16 years. If only, if ONLY I had really seriously tried to get off it back then. I even went to some meetings and read MA literature but gave up the fight and carried on smoking. If only I could do it over I would. It didn't get easier as time went by, one doesn't necessarily just grow out of it when one hits 30 (as I used to think). I'm here at 38 giving it my best shot.
I've struggled with lots of anxiety (my posts are full of it). BUT I've been assured that the anxiety will pass and I believe it. Anxiety is not really so evil, don't let it rattle you. There is a whole lot of adjusting to do when you quit and the brain chemistry, or whatever it is, does not just snap back immediately as you know. I bet it will be easier for you now than if you wait until my age. Give it up, hold on in there, it will pass.
I've started making notes on my calendar of how I feel each day, to remind myself it's only been 6 days/2 weeks/whatever out of my whole life, surely I can handle that tiny period of discomfort, which is nothing in the long run. I avoided the discomfort before and therefore have suffered a different kind of discomfort my whole adult life - lethargy, paranoia, not living up to my potential and all the other things related to being a chronic stoner. Anyway...