Marijuana/Weed smell around baby

Hi guys! My brother works at a medical marijuana growing facility and when I visit him at his house it smells very strongly like the plants, sometimes he smokes in the house too. Just wondering if there could be any negative effects for my baby. We go over and visit about every other week. Thanks!