Sugar pill week


I am newer to birth control and was curious about the sugar pill week. My pack has 7 of the sugar pills but my period only lasts 3 days of that. Should I be concerned about pregnancy? I have sex without a condom with my boyfriend and I can't help but have it nag at me in the back of my mind! Does not having that hormone pill the other 4 days leave me unprotected, or throw off my system?