My thoughts are that you have done the right thing, you have been the voice of that little girl and perhaps others who could not speak out, and now let it go and move on to better days ahead for you.
Living in fear can make us crazy, I know because I lived in fear for years.
If you need to move to keep yourself safe, then move. Your relationship with your brother sounds toxic and dangerous so there is no need to stay around.
If you need to seek out safety, call a women's shelter, they can offer help and suggestions even if you do not go there.
Hugs