Broken condom

So this weekend me and my BF had sex and twice the condom broke (got a new one each time) but he said that he never came and there was never much of anything in the condom. Should I be worried? My fertile window starts in a day or 2.