I'm worried I made a mistake

So... I'll tell it like it is!

For purposes of context, I live 4 hours away from my partner and we see each other every few months for 2-3 days, it's rough but I'm sure you can imagine the sex drive. 

My partner is always very safe when it comes to sex, he always looks after me well and doesn't let the fact he's using a condom take away from the experience. 

A few weeks ago I went to visit him and as always we couldn't help ourselves and had sex which is against both our parents wishes but we've been doing it in secret the past few times we've met (this helps explain things in a little bit). But we ended up having sex, protected of course with a few excursions along the way, but we were celebrating my birthday which was the following week while we were together, as much as I feel bad about it, I can't help but crave him cumming inside me. Now, here's where i mention I take AVA20 oral contraception pill to help balance my hormones, I researched and did all I could to prove to my partner it was okay and I wouldn't get pregnant. He was hesitant and looking out for me and us both knowing the consequences if I ever did get her pregnant, there would be nothing we could do and we'd be in big sh*t. I digress, he gave me the most amazing sex I've received from him, sorry about the TMI, and it ended with a pretty full stomach so to speak. 

Here's where I'm worried... And the technical stuff:

I was on my last row of pink active pills when we had sex and he came inside me, the next week I had my period from the yellow sugar pills. I can't stop freaking out and worrying what if I am pregnant now... My partner keeps telling me he made a mistake and is going to ruin both our lives from this and it's turned from something magical and special to a mental hell. 

We can't go to our parents, I can't go to the doctors by myself to even think about going down any path... I'm afraid...