Sorry you had such a rough experience.
I did IOP for 6 months after I got out of rehab. Like you, I was always one of the few people who were there out of choice, and not because they tested positive at work or they were court ordered.
Here's the thing. People come and go in these programs. Just like the revolving door of rehab and AA. The group dynamic constantly shifts over the weeks and months as newcomers arrive and people drop out or "graduate" (I always hated that term - how does one graduate from being an addict/alcoholic? I never graduated. I just kept coming back until my insurance cut me off).
So yes, I've come across the dominators, the racists, people whose vocabulary was limited to just a few obscenities, and my personal pet peeve, addicts and alcoholics who were convinced they didn't have a problem. But I also came across some really decent people who wanted sobriety as much as me. There are a few people I met in IOP who are great friends today. I see them in AA all the time. They stayed sober like me.
If I were you (and I'm not, so obviously do whatever you feel is best), I would give it a few weeks before you bail. Maybe there are people in your group that you might really connect with. Maybe you'll be able to help out a newcomer who comes in feeling like you did.
But yeah, I guess I was lucky to have a counselor who kept things on track. Anytime someone started straying off topic (there was one guy who loved to talk about his cars; it drove me crazy), she reeled them back in. If this guy can't control the group, I hope you can find someone who can.
Individual therapy has its merits, but in my humble opinion, in early sobriety, it's really helpful to have people around you who are dealing with the same stuff as you. People you can hopefully make deeper connections with. Not everyone obviously, but you may be surprised who might walk in the door each session.