Plan b help

Okay so me and my boyfriend have been having sex basically in regularity the last four times we didn't use protection and I've been taking the plan b afterwards so basically 4 times even though he hasn't actually finished inside. Should I be concerned about anything?