I don't know from what you wrote about what he's like he's giving you some pretty clear clues about himself. You said you feel like you are the only one giving and compromising well.........your feeling is right from what you wrote you are.
I say just let him go.
Ngaire