pregnant and a cat

an
I was told that when your pregnant u need to avoid cats but I can pet but nothing else. I believe that I can pet and cuddle whatever I want to and not clean the liter box. My cat is born at my house and I take him to the vet regularly and he's perfectly safe and clean. What's your opinion.