Just my personal .02, but if you can be reassigned to work somewhere else that means someone else capable can be reassigned to take care of the woman you are currently working with, right? The situation with this man seems very unhealthy for you and easily avoidable.