Cassandra,
It's so easy to jump to the worst case scenario. Been there, done that. I was at the point where I couldn't trust my intuition anymore--period, end of story. People would tell me, "what does your gut tell you?" and I couldn't say. I didn't know if I was speaking form paranoia or if my spidey sense really was tingling.
Just please go easy on yourself. I don't think it's too much to expect a call if you're going to be 4 hours doing a project that you (and he) thought would only take an hour. I think if you're working this together he has to understand that he's participating in this rebuilding of trust. Maybe I'm in the minority on that one, but I think he can take an active part in that process and you are not out of line saying, "hey, I need to know what's going on sometimes."
Huge hugs!