There are a couple of possibilities here. One is that it's your SON who doesn't want you to move in with b/f and he's using dad as an excuse. Not that he's consciously manipulating you, but we all have a tendency to want to have an excuse for raining on someone else's parade. Maybe it feels awfully soon to him--he's still processing the divorce, and now things are to be shaken up AGAIN.
Under the circumstances, I think I'd consider putting it on hold for now. You've said, yourself, that you are very close to your b/f's house and can spend lots of time together. If your son were moved out then I don't think his opinion would count for much, but he is still living at home.
You don't have to think of this as placating your ex, you can think of it as giving the kiddo a more stable life until he goes off to live his own life (more or less--he will still be home when school is out but he'll be much more independent).
Just my two cents.