I would have been tempted -- and probably would have -- said what you did.
That said, a better response is probably no response: "Oh really." Or "I hope you get it worked out OK." It allows him to keep his dignity and it keeps the problem/responsibility in his court.