Should I be worried?

I am currently on the birth control patch. I was supposed to take it off Wednesday and have my period but I took it a day late. It is now Saturday and I have no sign of my period. We usually use condoms too to be extra careful but there has been times we didn’t. I also took a Plan B not too long ago. Should I be worried ?