SO doesn't understand

My SO and I went back home with our son for the weekend. We brought back my sister for a week plus. Well upon us talking about something to do he brought up bowling and having someone he knows watch our son. We smoke and still take care of our son with no issue. But my thing is the potential person to babysit also smokes. I told my SO that I had no issue with her babysitting as long as she didn't smoke while she was babysitting our son. I've never, not once, seen how she is with kids, let alone high. My SO has and said she's really good with kids and I should trust his judgement. And he wigged out when I said she will not babysit our son if she is high or is going to smoke at all while watching him. I don't even care if he's already in bed for the night. It just will not happen. Opinions please?

If it helps no one has watched our son but me since his birth. He'll be 5 months old this month.