Maybe that is the reason it feels like a "power trip" as you say. Maybe need to better define the boundaries, for example, define exactly what HIS behavior is that you will not allow into your life, and what actions you will take if he does cross those boundaries.
There used to be an alcoholic, drug-addicted person in my life who I cared about very much. He, too, would call at all hours and talk about things that just angered me. In fact, just talking to him when he was drunk angered me to no end. I just could not eliminate my expectation that any person talk to me SOBER. At that time, I wanted to continue having him in my life, so I set very specific boundaries regarding the phone calls. It took some work examining my feelings around my interactions with him and my needs for my life. What it boiled down to was because I need to get up early in the morning to go to work, I go to bed fairly early. And I need to have a good two hours of calm and peace at night before I go to bed, in order to be able to fall asleep. I would get very upset when he called me drunk or high. Sometimes we would get in fights when he'd call. So because I could not predict what he was going to do day to day, and I did not want to take the chance of him calling past 6PM and wrecking my sleep schedule, I set the boundary that he was not to call me after 6PM and he was never to call me either drunk and/or high. I wrote it all out for myself for the conversation I was going to have to communicate the boundary (for details, see my post on Transformyself's thread about Boundaries) and even read from it when we sat down to discuss.
It worked out perfectly. He respected my boundaries. It was simple and consistent enough for him to remember, so whether or not he was going to drink, he made it a habit to call me between 5 and 6 PM. I think he overstepped the telephone call boundary one time only. And it was great practice for me because he was someone I KNEW loved me and cared about me. Of course, I waited for a time when he was sober to sit down and have the conversation with him. I was so nervous that first time, I was shaking. IMO, this experience is occurring with your Dad because he is a safe person for you to practice this skill with. I'm glad you have this opportunity!
(((hugs))) hope something here is helpful.