How many days late should my period be before I worry?

Me and my boyfriend have unprotected sex  and use the pullout method. My period is 4 days late, and I'm starting to worry, but don't know if it's time to start 100% freaking out yet.
How many days without my period should I begin to really worry that I might be pregnant?