I agree you need to talk about this.
I will, however, offer up this thought as well -- because I recently had this discussion with a friend who, at age 40, was considering adopting.
Having kids is very hard work.
Yes, I know it's obvious, that everyone knows having kids is hard work. But it's hard work on a level that you don't expect until you have kids. It's incredibly rewarding and amazing and makes you love in a completely new way and it's the most amazingly wonderful choice I ever made in my life.
But it also completely exhausting, 24/7 work that changes your life in ways you can't predict or imagine ahead of time. And if you've lived to the ripe age of 38 or 40 and gotten used to being able to do things your way -- the changes that come with a child, though mostly positive, are something you might not be prepared for.
I hear new parents say "I can't wait until Joey becomes a little older, this age is so hard." And I smile and don't tell them "You think it's hard NOW? Wait till they're 10... or 15... or 18... " because the truth is -- once you have a child, your life is forever changed. Your child becomes the primary and most important thing in your life -- ahead of your job, ahead of your spouse, ahead of everything.
Having a child is not like "adding something" to your life. It's turning your life and your priorities and your habits completely and utterly upside down.
Are you both ready for that?
Another thing that I would consider in your situation is that the idea of having children is very different from the reality. One of my kids has a chronic disease that has made me spend my parenting time about 3/4 on her and 1/4 on the other kids. Having "normal" children changes your life. Having children with illnesses and disabilities takes over your life completely. And it's a possibility that you have to take into account.
I know I sound really negative to having kids, and I'm really not. I'm not trying to discourage you from having children, or adopting a child, if your heart says that's what you both want. But I know that I completely underestimated how radically having children would change my life, and I know that I could not have handled a change that radical now (in my 50s) -- let alone ten years from now.