Cats ! Kinda long sorry!

So I have a cat, I've had him since he was 6 weeks old and he's 3 years old now. I'm 10w1d today and my mil is convinced that we need to get rid of him. Says that the hair he looses gets into my nose and to the baby. I don't change the litterbox- my bf does that now since we found out. Mil also says to get rid of cat especially when baby arrives due to the cause of asthma.. Everything I've read says it's okay to have the cat around. But of course my bf is taking his moms side on the matter. Any experience with this? Asking my dr tomorrow just wanted to hear everyone else's views.