Cats and babies...

A
I have a 5year old totally unaggressive cat and her daughter who's about 2, very needy (1 blue eye and 1 yellow btw, so cute) and everyone keeps telling me I have to get rid of them once baby is here. I don't understand and chalk it up to superstition. Is there something I'm missing?? My boyfriend has been on litterbox duty the entire time so aside from that, what's the problem?