Took pill late then had sex

I left my birth control at my apartment yesterday and I was sleeping over at my cousins house so I just took it when I got home this morning (It was the last pill in the pack btw).Then today, I had sex and he came inside of me. Should I be concerned about pregnancy? Should I take a plan B or will that mess with my birth control?