A domestic violence center or hotline will be able to help you. Your area may have a low-cost or free legal assistance program, to help you get restraining order, etc.
Listen to your fears, but they don't need to overwhelm you. He may just be a very manipulative but lazy drunk. A DV center can help you put a safety plan in place. Simple stuff like changing your locks and computer passwords can create a lot of peace of mind.
Maybe your family can say, 'any further communication with us should be in writing, or directed to our attorney. Thank you.' Or you can say 'please be advised that as of today, all communications with you will be forwarded to my attorney.' That might stop him in his tracks.
A restraining order can shut up a lazy bully. But for a certain type of abuser, it is like waving a red flag at a bull. A DV counselor can help you sort out which one he is.
Good luck to you!! It's not easy to have to be so strong, but you can do it!!