Newborn and litter box

I know all about toxoplasmas. My doctor said are cats are unlikely to have that because they have been in doors all their lives. My question is that we are stuck in a master bedroom connected to a bathroom that we share with room mates. We keep our cats in the room only and I am around 12 weeks pregnant. I'm stressing about when baby gets here and the baby being by the cat litter. My husband already does a great job cleaning it every day but I'm afraid that won't be enough when Baby is here. Has anyone one ever dealt with this or had a newborn live in the same room as the litter box? I will not be getting rid of my cats they are my babies and I am not looking to be put down for this topic. I'm just looking for advice so please help me! We are in the process of trying to separate the cats space from our space in the room. Our room is probably around 600 square feet.