He wants his mom there. He doesn't care that you don't. That's what his ACTIONS say. His words don't mean anything. You have every right to feel sick and angry. It's upsetting to you because he isn't listening and he is showing you he does not care how you feel.
He is getting his way by degrees. Taking baby steps towards your boundary and you are stepping back in baby steps to accommodate him. Your "rules" don't mean anything to him.
You say you feel like telling him you are done and you are leaving. Are you done?