My cat and being Pregnant.

Jennifer

So I am recently 4 weeks pregnant with our 2nd (rainbow baby) and my doc told me Its dangerous to have a cat since they carry so many diseases that can affect me in my unborn child I understand that I am not allowed to change the litter box so my husband does that and my cat is indoor only and has been since birth also is updated on shots. Do I have to rehome him? Or is it okay as long as I don't change litter box?