Marijuana and ttc?

Anyone had a hard time getting pregnant while your partner smokes pot? I don't want to have to tell him to stop but I heard it lowers Sperm count. Anyone gotten pregnant while partner smokes pot?