Visitors are not what motivates an addict to get clean. Maybe he needs the reality check of no one showing up on family day to realize "damn I lost everyone by choosing to live this way" in order to finally see the full consequences of his actions.
I also agree with others that a rehab is not a place for small children. Children take something away from every experience they have, who knows what she will take away from that visit.
He has consistantly put himself and drugs above everything. Although we like to think that an addict would choose sobriety after seeing their precious small child, it just isn't that way. For four years now he has known that to be a father he had to be clean and sober and for those four years he has choosen drugs over his child. We all wish it was different, but sadly addicts don't become sober because they have small children that love them.
Whatever you decide to do, don't make the decision based on him. Base the decision on what is best for your daughter. You never know what you will run into when you go to a rehab center (patients in rehab centers have out bursts, can get violent, say inappropriate things, ect.).
If you decide not to take her and he later tries to claim that he didn't stay in rehab or didn't remain to stay sober because he got discouraged because no one came to visit him and he didn't get to see his child well then he is just using you all as an excuse to keep using. Visitors aren't what give addicts motivation to stay sober, the addict hitting bottom and realizing that a life of drugs isn't worth it and that they deserve a better more fulfilled life is what motivates them to stay sober.