Old wives tales : CATS

Am
I have two cats and my grandmother and SO mother tell me to be careful because they could smother my baby. How many people has this actually happened to and hasn't happened to? Should I be worried