You're in a tug of war with your 2 brothers over this. As long as they can get you to engage, to pick up your end of the rope and pull, nothing will change. The "why" may never be answered, but at the end of the day it doesn't matter. They don't like your wife. It's really their issue, which is evidenced by the fact that other family members feel differently.
Let go and let God. Don't waste time justifying yourself to them. Get off their merry go round. Let go of the anger, and focus on the rest of the family. Just my 2 cents...