Condom slipped out

My boyfriend and I were having sex and once we finished I went to get off and the condom slipped off from him and got stuck in me. It seemed to fall all on him and not in me but I can’t be 100% sure. I’m on birth control also and have been taking it on time but earlier in the week I was late by 2 hours on 2 consecutive days. After that for the last three I’ve been on time. Should I be worried? What should I do. Since I’m already on the pill I don’t want to take plan B and get all messed up. Should I be ok?