I’m on the pill and had sex without a condom

I’ve been on the pill for 4 months now and I have never missed a pill. I have been really good at taking them on time however on this past Sunday I took it 10 hours after my regular time but i still took it on Sunday. My boyfriend and I had sex Tuesday night and I felt confident so I told him we could have sex without a condom. He pulled out way before he finished and finished outside of me. Now I’m worrying. On Tuesday I had started the last week of my pack and I only have 4 inactive pills. I’ve been getting cramps like I’m about to start my period but should I feel concerned?