The pill and sex

Me and my SO had unprotected sex. I'm on the pill and he also pulled out. We have done this before without him pulling out and I never became pregnant. Just wondering your opinions and if this was you would you be worried ? I take my pill at the same time everyday and usually use condoms too but you know these things happen sometimes :P thanks in advance :)