Liz....your feelings are valid (all feelings are). You have a right to them....and, I am glad you aren't stuffing them.
I have a hunch that there is a bit more, under the surface than JUST because he can't say "I love you".....
An intimate interpersonal relationship is complex..with lots of nuances. Only the people actually in the relationship can know what they all are.....
In my own experience, anything that is "niggling", early on, in the relationship--doesn't go away...it just gets bigger, over time--especially, in times of stress.....
Liz...be true to yourself....