Birth control pills !!!!! Help!!

So I've never taken birth control pills I got some but with my inexperience I took them backwards, I started taking them the first day of my period which was January 6 but I started off with the inactive pills!!! I was supposed to start with the hormone pills, and I been having unprotected sex this whole time, would it really matter that I took the inactive ones first? I didn't have sex at all during my period which is when I was taking the inactive pills, I started having unprotected  sex when I had already taken at least 4 hormone pills.