You stated a boundary, then when challenged you came up with a plan B if you needed it (leaving). I'd say you did just fine
Yes, anger and manipulation is a common reaction. You're no longer maintaining the status quo and he doesn't like that. Be prepared for him to test you again.