If I've learned anything from my own experience with AH's dysfunctional family, it is that they can be fine when you are doing what they want you to do. But, try to change the family dynamics and WHAM, they turn on you like a pack of wolves. I've decided to go no contact with those that are trying to blame me for where AH is today and only invite positive people into my life. The people who know and love me know what I've been through and understand my need for it all to end.
Not to tell you what to do, but I would say nothing to the grandparents. Anything you say will be twisted and used against you. Let them approach you regarding visiting their grandchildren and you can pick the time and place (not at their house).