Cigarette smoke and infants

I am not a smoker and no one smokes inside my house. I have an almost 3 week old and have family members who smoke. What precautions do I need to take aside from the obvious of them not smoking around him? Also in the past I would have an occasional cigarette socially and am craving one since I haven't had a drink and feel that my baby's feeding schedule is not in much of a routine to do so yet, I breastfeed, and know that smoking often can affect your supply too. Many things I have read just say to quit smoking but I'm not in that situation, I'm not addicted I just want 1 here and there. If you smoke how will you be addressing this? I know to wash my hands and brush my teeth afterward as well as change my shirt. Is there anything I'm forgetting? Is it necessary to take a shower also? I don't want to put my baby at risk or let him breathe in any toxins. Thank you in advance for any advice!