When my children were very young, I did not allow my parents to see them alone, no babysitting because, as I told them, I would not leave my sons with a couple of drunks. That was my boundary. It was never violated. I did it to protect my sons.
You cannot control anyone but yourself. If your mother chooses to cope with her life by not talking about the elephant in the room, that is her choice. You cannot, and IMO should not try, to make her take a choice that you think is the right one. You are all adults here. Perhaps in time your mom or your step father will choose to deal with the sutation. Perhaps not. But that is not yours to control.