No, you didn't "mess up"--you were uncomfortable on your "date" and there is no requirement that you be friends with him. It's OK, you haven't been harmed (apart from what you have been putting yourself through), he hasn't been harmed.
This is a non-event, but for the importance you are giving it.