Thousandword, you knew this was coming before he did. So he may be trying to understand this or he may be trying to manipulate you.
You explained yourself at home or maybe in counseling. He and others will hear, understand and do what they want. Your life is dependent on what you want. Stick to your boundaries.