Plan b?

Ladies I just started taking the pill last Monday and you are supposed to wait one week before having unprotected sex. Well Sunday night my boyfriend and I might have jumped the gun. Should I be concerned or am I just worried for nothing?