children thrive on consistency and clear boundaries. when "daddy" just shows up, unannounced and/or without previously arranged for supervision, it creates chaos and unrest for the child.
remember the jack in the box toy.....you'd spin the handle, the tune would go, but you never knew WHEN "jack" was going to pop up - and it was always just a little scary.
not only does the child need a reliable schedule of visitation, but your ex needs to know that there are RULES in place, and that they benefit the CHILD, not him. your ex is still all about him.