serious question!!

Please no bashing this is a serious question! My son is formula fed every now and than I'll give him breast milk that I've pump and froze right after he was born! When I give him a bath I sit in the tub with him he seems to enjoy bath time better (he's 3weeks) I'll hold him before I wash him other wise he sits in his bath seat. Anyway, today when I was holding him he calmed down (i think it was the skin to skin) but he starting like searching around for my nipple and found it and he instantly latched on! No big deal! we never had that problem I chose to formula feed because it will be easier in the long run for us because of how my life is! Anyway I smoke marijuana in order to eat. I take one hit a day mostly at night just before dinner. So my question is how much of an effect do you think it has on my son? am I a horrible mom? It's not like he is EBF it's more of a if he finds it and latches I let him eat till he pulls himself off. And he's only done it twice and both times has been during bath time or when I do skin to skin to calm him down! I know people who have EBF and smoked constantly and there were no issues. My MIL is the one who seems to think it's a horrible thing and makes me feel like the worst Mom ever all because he latched on to my breast twice since I've started smoking again! 😭😶😕