It does sound harsh. Is this the first time you have tried to quit and told him?
If it is I think maybe you should try again tomorrow but if this is a pattern you may have to take some time to prove it to him and to yourself.
My wife is becoming more confident in me as time passes but ay first it was I hope you can but I have heard this tune before.