Cat liter

Lisa • 36 years old, first time mom after 7 years ttc, baby boy Colton born Aug. 23rd 👶. Lost 2nd baby at 7 weeks.
My dr says not to change the cat liter. I've heard of friends who cleaned cat liter pregnant with no problems. I live alone and was told to use gloves and wear a mask. What does everyone else's dr say about this?