NurvaRing and Sex

I've tried many bc methods. IUD didn't fit, Nexplanon made me bleed 5 days of every week, and I couldn't remember the pill. My doctor put me on NurvaRing. And its been working wonderfully. Until recently. It fell out. I have no idea of when or where. My boyfriend and I got freaky at least once when it was out. He doesn't want kids and I'm terrified that I'm going to end up pregnant. I can't remember for sure what week I'm on but I believe its week 3. It hasn't been out for more than a day on week 2. Plan B won't work because I'm 30lbs over the weight limit for it. Is there any need to worry?