Does marijuana make TTC harder?

We're trying to have our 2nd baby, but my boyfriend feels i'm being too controlling when I asked him to stop smoking weed until I get pregnant. I read that weed can cause sperm to be less mobile for reaching the egg. And hence may be why Im still not pregnant after 3 months of trying. Am I being selfish or is he?