Here's a different spin. One of my aunts and I were talking about this type of thing a few months ago. She has been married for over 30 years, and met her husband while in the process of divorcing my uncle, or maybe prior to. Not sure, but she knew she was going to divorce him, that much I know. So I asked her if she thought it was wrong to have a crush on someone else while divorcing?
She said she did not advocate an affair, even though that is what she did, but she did advocate the crush. She said that the feelings a crush brings up were a reminder of what was missing and what one was breaking free to find - happiness and joy of living. She has seen too many of her friends forget why they were getting a divorce, go back and continue to be miserable, but those that had "feelings" managed to get out and had the chance to create the life they deserved and did. In her opinion hanging on to a crush was sometimes what people needed to keep them focused to get them thru to the end of a divorce- even if they never ended up dating or in a relationship with that crush.
I thought it was an interesting perspective. Different than the "norm" in a way.
And by the way, I don't think you're a terrible person or scum or any of those things. I too know the lonliness of not having a partner.
Good luck to you -