What does your custody order state?
In my country the order is in place until the age of 16.
16 would be a much better age of maturity to let your child decide.
Can you explain to your child that when he is 16 he can then decide who he lives with?
I would definitely have concerns over the very loose rules in the other family.
Schooling should also take a top priority & sounds like you're right on the ball with that.
I am so sorry you are having to go through this.
I am reading a book at the moment on how to co-parent with a toxic ex & it addresses children caught in loyalty conflicts & I can't help but wonder if this could be your son. Is your XH feeding him the will to move in with him?
Have you noticed changes in behaviour & negativity towards yourself?