Should I worry!?

I’ve had the implant for 3 years. It’s overdue t come out because my gyno keeps pushing it back to take it out. She gave me pills as a substitute birth control 4 months ago but I ran out. She won’t call out more pills. I’ve now been out of birth control for a month. My man and I had sex a few days ago and he finished inside me.... no condom of course.... definitely not planned. I am freaking out! I’m in my “fertile” week currently. I’m not even close to finding out what the damage is, but should I be worried? Should I just let it go?