Boyfriend can’t keep it up?

So my boyfriend and i have been together for a little over a month. We have had sex about 4 times and it all starts with a condom on and then because he gets soft the condom breaks and he wants to proceed without it.. it’s really nervewrecking for me because I’m not on any form of birth control.. but oddly enough the first 3 times he didn’t ejaculate at all.. which i thought was weird.. but last night when i was over his house he did.. he pulled out but I’m still nervously. Should i take a plan b? And is there any reasons why he might not be able to keep it up when it’s time to put a condom on?