Help! I’m worried

So my boyfriend and I are long distance and throughout this cycle, I’ve only had sex with him once.

It was protected and he didn’t even cum inside the condom.

My period is late but I woke up to brown discharge. My cycle also might be off because I got off of birth control 3 weeks ago.

Should I be concerned??