Is it normal to hallucinate more when pregnant?

It seems like I'm always seeing things that aren't there. I just entered the 2nd trimester and it's gotten to be more frequent. I'm worried if I ask a doctor they'll take the baby away.