Birth control/ovulation

I’ve been on birth control for a while. However I had a pregnancy scare and was told to go off it. Had my period but was told to stay off it. Was off it one week in addition to blank week. Based on my period I should be ovulating now but you don’t normally on the pill. I went back on it Friday. Should I be concerned about having had unprotected sex plus twice where he only pulled out around what based on period (not factoring birth control) should be my fertile window? Posting in two groups.