How about just telling him the truth? That you are uncomfortable with that 'routine' and therefore will not be going with him?
It's not your bag. Obviously going and then resenting is not any good. Just tell him 'no thank you'. If he is the person you think he is, he will understand.
J M H O
Love and hugs,