Dear Terrispots, a fundamental problem, now, is that you and your husband are not on the same page. As I mentioned before, if you two are not absolutely united and back each other up--this will never get off the ground.
Can the two of you go to a separate place--where you can freely hash this out until you come to a united policy? It might take quite a lot of gut level discussion (LOL)---but, there has to be a point of agreement between you.
sincerely, dandylion