I need justification

So my boyfriend and I had sex like right before my period. I'm late and during that time we had sex again. Now it was protected though I'm not on the pill, and he says the condom didn't break. I  tend to have irregular periods, which are very hard to keep track of so I kinda don't. I get very easily stressed and have missed peirods before. I'm just low key freaking because that was our first time not just as a couple but first time. So right now I just feel like it'd be great for someone to tell me don't worry about it until next month. Also I haven't mentioned anything to him about it.