I took it as a boundary lesson and not anything else.
Here's the example of my neighbor's and my conversation:
She: I noticed you haven't come by lately like you used to and we really miss seeing you.
I: I know. I miss seeing you guys too. In setting up my new healthy family life, I've noticed I'm uncomfortable around any kind of drinking or anyone who has been drinking. I hope you understand.
She: Of course I understand! I hope we can figure out a way to see each other soon.
I: Me too. See ya!
And that was it.