Lurkings,
I think people are either physical or not and it´s not unnatural to feel like you feel. I know a lot of healthy people who are just not into hugs or touch. I like to touch and be touched up to a certain degree, and it also depends on my moods. After difficult rejections, I cannot bearto be touched. When I feel content, I like it.
The important thing is to be true to yourself and set clear limits without offending. You seem to be able to do that. What I find difficult is when people walk into my zone of distance. They come near, and I go backwards, but they just invade my territory! I have real problems with that one.