Plan B

I have never taken plan B in my life until last week. Long story short. I was on birth control (depo shot) for 4 years, my last injection was in June due to the horrible side effects. Anyways I had sex with my boyfriend and he came inside me without thinking, I took plan B at least 10 hours after and then the next night he did the same thing. I took another dose of plan B since plan B does not work as total birth control just prevents implantation of a fertilized egg in the uterus by altering its lining. On my calender it states I am fertile.  Not sure if I should keep stressing myself or not.