2 years is a long time in limbo land hoping for what once was or what we want it to be, again. That he smokes is neither here, nor there.
Sounds like he is not into the kind of relationship you want, right now, just as you are not into the kind of relationship, he wants, right now.
If the dinner makes you uncomfortable you can decline and say one of the kids is sick or simply tell the hosts the truth. It's unfortunate but it happens.