The pill & not pulling out

Hey, I’ve been on the combination pill for years, always taken it on time, but a couple of days ago I missed a pill and doubled up the next day and yesterday me and my boyfriend had sex. He came inside me and we continued having sex for a bit. I ended up taking a plan b later that day just to be safe, along with my normal birth control. I’m just wondering if anyone else has gone through this and if I should be worried?