Smoking

Okay me and my significant other got into a fight last night because his father lives in a trailer house and him and his wife smoke cigarettes in the house one after the other. (The whole house is filled with smoke 24/7) It’s so bad that I can go in the house for about 5 minutes and come back out smelling like cigarettes terribly without them smoking around me. Anyways I told my s/o that I don’t want to bring our baby there once he’s born and that if his father wants to see our baby that he will have to come to our place because I don’t want to bring my baby into a house that reeks of cigarettes. He didn’t like that idea but honestly I didn’t much care if it hurt his feelings because his father hasn’t been in his life since he’s been born anyways and only uses him for money or if he needs help with something. Am I completely wrong or being too overprotective of my child? I don’t want to be insensitive or come off rude but I don’t like people smoking around me much less my baby.