Pregnant¿ I hope not, help:///

Alex
Me and my boyfriend have been having sex regularly for like the last month, and he always pulls out, but this had happened before, where I missed my period by like a week and I wasn't pregnant, but I've been told that when you don't have sex for awhile and then you jump back into it, it kinda messes with your ovaries and is like oh this is different and throws it out of whack. He's worried and it's only been 3 days, should i be worried?