so scared

i know im about to get a lot of mean comments and rude comments but i had unprotected sex with my boyfriend Thursday, January 2nd i took a Plan b the next day, Now what im scared about is we had unprotected sex again this past Sunday, he always pulls out but this time hes not so sure if he pulled out on time. I know im not suppose to take plan b on such close dates but will the pill still affect me? Or am i screwed? (getting on birth control soon btw)