Kudos to you for being able to separate the feeling from the person! What a big step forward!!
I had to learn to treat these kinds of feelings as a 'road map' to what I wanted and needed out of life.
When I missed being hugged or held, it wasn't that I missed the real him, or needed/wanted to go back to the ickiness of him. My mind was playing tricks on me.
It was simply my body telling me that one of the things I love in life is being hugged and held by someone who cares about me -- who loves me for me. (That was the illusion I had sold myself for years: that I was truly and deeply loved.) I found that going back to 'him' to satisfy this craving was terrible beyond words.
I had to use the road map to help me move on and (eventually) find a healthy situation where I could be held and feel good about it.
But there was a period where I did not get to have what I wanted (the holding). That was very hard for a while, but got easier when the "withdrawal" wore off. It also helped to build different kinds of touching into my life -- like regular hugs from kids/relatives/friends, partial or full massage, even going to the beauty salon and having someone fuss over me -- all of these things helped with the fundamental need for human touch, and they made me less and less likely to go back to the toxic source.
Sending you hugs of a different kind, maybe they will help you stay on the healthier track:
GL