I know that summer day care is quite pricey, but I know about another mom that I work with who's ex H was trying to do the same thing with her kids. He didnt want to split the day care costs, he wasnt thinking really about the kids well being, he was thinking in terms of dollars and cents.
Her day care sounds like a wonderful program that is organized with alot of field trips and summer activities, her kids actually like going there most of the time. It's like day camp to them, swimming, field trips, oh just all kinds of things there. This is really good for her kids, consistency, she never has to worry about them not showing up.
He may not understand completly at his age the dangerous situation he would be in, As his parent, you are not always going to be the good guy, and one day that little guy may thank you for caring enough to not go along with his father's wants, and his wants.
You do what you think is best for your child.