Unfortunately there is no exactly "right" answer to this question. I supervise my son's visitation with his son. I don't think he would purposely put my grandson in harm's way. But drugs make people do things that they wouldn't otherwise do. It clouds their judgement. Luckily, my son's ex-girlfriend has been very cooperative in allowing this arrangement to happen.
If my son proves over time that he can do things responsibly.....like have a driver's license, valid insurance, and remains drug and alcohol free. I would have no problem with him spending time alone with his young son. Until all of those things happen, he has to deal with the relationship in terms that the healthy people in my grandson's life dictate. I am fully and 100 percent supportive of my son's girlfriend's decisions with regard to her son's life. Right now......she is the sole responsible parent for that little guy and she deserves my respect and cooperation.
I hope that you are able to make arrangements that keep your child's best interest foremost.
gentle hugs