I think taking him out for lunch next week is a good idea. I know though that he will offer to pay for me tonight - he's already mentioned it a week ago when I mentioned how expensive it is.
I'm really anxious at the thought of calling him and saying no to the dinner and hearing the hurt and disappointment. Time for a cup of tea and to refocus and calm down.