Call me contrary but I'd be pretty peeved if my significant other refused to have lunch with me during the week on the grouds that s/he's too busy, and then had the time to throw a party for someone else and used lunch time to do party prep. If you're doing meetings at lunch that's a whole different thing and more understandable. The no lunch/party thing would make me feel like I was not particularly important to my lover.
I don' t know your whole story but in a vacuum here it seems like the issue is not so much control, but a relationship in which neither party really feels comfortable. A good love relationship makes you feel good, not bad.
Just one gal's opinion.