Couldn't you just say "hello" politely but in a very detached manner, and not stop to chat? Kind of like how you greet someone you hardly know?
Or do you think he'd force you into conversation?
Sometimes when we dread these things, they turn out better than we anticipated.