Hi, Butterfly. I think you were completely within your rights to pick up your son and deny the visit. IMO, you're also within your right to decline a make up visit. He missed the original visit pick up, you're under no obligation to further tweak your schedule to give him an alternate schedule.
DS was 6 when I filed for divorce and that's about the time the issues with AXH's visits (and no-shows) came to a head. F... it was hard on DS. (It was hard here, too, but I'm not a little guy who wanted to believe his dad hung the stars only to realize Dad's priorities were elsewhere.) It's impossible to completely protect our kids from their A-parent's behavior, but we can definitely let them know we're there for them and help them process stuff. ((((hugs)))) to you and your son. You're doing a good job.