What would I do would depend upon how fat my wallet is.
If I could do without the money...my peace of mind and getting on with life sure would be worth dropping the whole thing and just moving on.
But when we are talking a few thousand dollars... well I would need worry about my needs for the future and a peace can come from knowing that is cared for as well.
The answer is simple... He wants the house, he takes out a second and pays you off your share. If he can't...the judge says sell or go to jail.
At what cost would you put on peace of mind? Only you can answer that question for yourself.