I agree with dandylion- I would not allow him to come over, especially if he is still drinking/having mood swings. Being in a public place may be safer option.
As far as phone calls, you can set boundaries as well if its too much. You can have a life and don't have to be home every night for the calls either. If he is drunk, then I would not allow him to speak to DD. If she is only 3, she probably wouldn't care if he called or not every day either. I know my DS who is 4 doesn't care if he talks to his dad every day or not.