The Pill and Sex

I’m on birth control and I’ve had sex a few times with a condom but it broke once but we caught it right away and changed it... is there anyway that I could be pregnant? My boyfriend always pulls out. A lot of people say I can have sex without a condom on birth control but I’m still too paranoid I will become pregnant..??