I also think that the COURT respects parents who assert their rights (in an appropriate way) early, and as often as is reasonable/necessary. Some people are just plain litigious and love to drag their ex back to court at every opportunity (some of these are people who are NOT alcoholics or abusers--they are just plain angry (often without good reason) and are the same kinds of people who would sue over other frivolous stuff).
HOWEVER, courts also look askance at people who "sleep on their rights"--doing nothing to enforce them for years and years and then explode in a conflagration of litigation. So taking someone to court every time they drop off/pick up the child five minutes late is annoying to them, but so too the litigant who has tolerated violations over and over for a long time and suddenly seeks to enforce an order.
If a lawyer says you don't have a legal leg to stand on, then you have to go to Plan B. But if you DO have a case, then you have to weigh the cost of fighting over it (economic and otherwise) vs. your likelihood of winning and how important the issue is to you (and your son). These issues are rarely no-brainers, but you can't even begin to decide till you know where you stand.