11 days on the pill too soon?

I'm now on birth control pills, the doctor told me not to have sex the first 7 days of taking birth control, I had wanted to wait way longer than that but I was stupid and had sex on day 11 of the pill, with a condom and the pulling out method. I know the doctor said 7 days but I still feel like 11 is way too soon also. Should I be worried that I could possibly get pregnant. I don't know, I'm just worried and would like to hear some answers. Me and my boyfriend are in a long and serious relationship, we love each other very much so please don't judge