Will I be ok?

My boyfriend and I were having sex and we were using a condom but during it apparently the condom came off and got pushed inside of me. I got it out but he said that he came in it a little bit. I took plan b the next morning. I am not on birth control. Should I be worried?