We often say here to look at someone's actions more than their words. What you relate is a man allowing you a lot of access into all of the facets of his life - including his children.
You wrote a while back Liz that you were anxious about saying, I love you. Maybe you could journal a bit on why you felt it necessary to nail this relationship down. Was it a control move on your part?
As for 'pulling back', I see the worth in focusing back on you. Don't you dislike your job? Maybe its time to update your resume and network. As the job market comes back after the holidays, you are ready and able to whiz off a bunch of applications one night a week or have lunch with someone has works where you want to work or has the job you want someday. Invest some time into your economic future. Take that emotional sting and use it for your own good.
Take care Liz!