Should I be worried?

I lost my virginity to my boyfriend on the 15th and we had sex multiple times since then. I've been on the pill for around 3 months now and have been taking it at 4:30 everyday (maybe 10-15 minutes late/early some days) and have never missed a pill. We didn't use a condom (100% both clean, we were both virgins and got checked before just to be safe) because he had trouble getting it in with one on but he still pulled out just to be safe. But now my period was meant to come on the 20th and I still haven't got it (I'm very regular). My boyfriend has begun to worry as we are both young and don't want a baby right now. I am also again abortion so I would end up keeping it as I don't think I could adopt it. 
But should I be worried?
Should I take a test?
If so, when?
Should we try to use a condom again?