advice.

My boyfriend and I had sex for the first time ever with eachother, I myself haven't had sex in a long time (like half a year) and he hasn't in a long time as well. My reasoning is because I'm very neurotic and think of the worst possible outcomes, really don't want to be pregnant... so the anxiety kind of ruined sex for me. We used a condom but when he tried adjust it after it riding up or something it broke and he put on a new one... but he put it backwards in the heat of the moment and then proceeded to put it on the right way again. The whole situation sketched me out a bit so I went and got Plan B One Step. I guess the advice I need or question I'm asking is, should I be worried or are the chances practically non existant...?