Weed...

So my significant other is really into smoking weed. He likes to do it to relax. We got together 4 years ago and he said he used to do it all the time. Until he met me. Long story short. We have a baby girl on the way in Oct he had quit smoking weed am I overreacting I get so mad when he does it and idk if I should just let it go or be mad. He doesn't do it around me he does it on his own time and only takes a hit. I need help. I know he enjoys doing it but I don't want that around our daughter and he completely understands and has stopped. But should it be a big deal???