I am uncomfortable being this blunt....but I think the last thing on your mind right now should be your relationship with your boyfriend and the SIREN that should be going off in your head is the very real risk of your daughter being raped.
A famous priest who works in the addiction field has said that incest is the unspoken dark secret in addiction.
Your daughter is not even the insane brother's blood relative. So what will he do to HER?
Why you are allowing this man--who is behaving like an animal and making "gestures" toward your daughter--anywhere within 100 MILES of her is stunningly inconceivable to me.
Your focus is on the wrong thing. I suggest you wake up. Get him out of there. And if it kills your wonderful relationship, so be it.