Elsie, in my opinion it is still all in the response that you are giving both verbal and physical cues that your children will pick up on. Your inner turmoil could show through no matter how pleasant your verbage. Working on acceptance and detachment helps to ease those inner butterflies.
Children may misunderstand your "Okay" as an affirmitive response in agreement, you're right. Would a lack of response at all be more appropriate for you? When he starts in with nasty behavior in front of the children tell him calmly but firmly that you will speak to him when he can be respectful. This is something you would tell a child acting out so you're kids should read this literally. Once stated, any continueing nastiness can be rebuffed with silence. If your kids are confused remind them of what you told him the first time.
I delt with some serious bullying as a child. I won't go into details, but I remember well my mother telling me to say this to them. When I would tantrum or be agitated with her, she would say the same thing to me, that she would speak to me when I could be respectful to her. I made the connection young (preschool age) that my behavior was bullying to my mom. I think your kids could make the same connection easily that his behavior is bullying and you will not engage him.
Detachment is tricky business and can flame things at home when addicts feel you taking a step back from the same old confrontations. Tough stuff. Stay strong!!
Alice