Ok you have clarified some things.
What it sounds like to me.... Sounds like it can be addressed and should be able to be solved. The finances they need to be addressed. I don't really understand the thinking of "rental" payments for children. His are there 50% of the time, your son is there a little more. He has more than one child though and you have one. This to me seems to be splitting hairs in determining what is financially equitable. The whole "well I am paying less living here than I was" is not a substantive argument for you to pay equal to what he does when you make so much less. The mortgage stays the same no matter how many people live there. I might understand a financial difference in food but at the same time when you live together is it always to be down to the penny exact? Are you building a future together, or only a future together except financially? I so get each of you protecting your interests because you have been through a divorce. It really does not look fair, even close to fair to me.
As for the "I love you". You have a lot here that is really, really good. You may find someone who says I love you to the metric you feel warranted, that does not have the other pluses that this guy comes a long with.
I'm not sure that I perceive breaking this up would be warranted given that most of what you write about him is really good.
I'm confused I guess.