Having sex on just birth control?

So currently I'm on birth control but when my boyfriend and I have sex we always double down with a condom just because neither of us want to have kids this young. However recently both of us have kinda been wanting to try without a condom just once and I was just looking for advice on whether we should pull out or if any of you were having sex only on birth control and what you guys do? Just a back ground I've been on birth control for more than three months and it's a levonorgestrel and ethinyl estradiol .1mg/.2mg perscription. My mom told me she and my dad just relied on BC for years and never got pregnant but my boyfriend's family has a history of strong babyjuices (his brother has seven kids at 25)