Take it slow..listen to your gut...if your relationship is meant to last a lifetime you will have a lifetime to work on it and to be together...
I am currious as to why he does not visit you instead of you going there? Maybe you could take turns going back and forth?...