Pill anxiety

I've been on the pill for like 5 months and it seems to make my anxiety worse, but I'm worried if my bf and I decide to have sex even with a condom I'll get pregnant, so I'm scared to stop taking it even though I don't like the way it makes me feel. Is this normal? Is there a way to lower my anxiety and not stop taking the pill?