BC FAIL

Been taking my birth control a little sketchy. For about a week I take it everyday at 10pm but I would take it at different times the latest 12pm next day I wasn’t worried because I wasn’t having sex. But one day I took it around 6pm next day and had unprotected sex that same day should I be worried?? I’m on the combo pill