4th week on the pill

So my doctor told me my birth control, I take vylibra, would start working on the first week but my boyfriend and I wanted to wait a little longer before he actually nutted in me just to be cautious. We don't use condoms but I just finished the 3rd week of my birth control and let my bf nut in me. The doctor said after a week it should work but im nervous Bc many say to wait a month. Even tho it was technically almost a month am I fine? I take it everyday within the 4 o'clock hour usually exactly at 4 sometimes 20min after 4 but always in the 4 o'clock time period and I haven't missed a day.