advice please?

My boyfriend and I had sex yesterday and we used a condom whenever I rode him I went the deepest I've ever went and we seen cum at the base of him we both know it was me. But when he finished he made sure even though he had a condom on he didn't cum while he was inside  after we finished he went to the bathroom and checked for leaks and made sure there were no tears and there wasn't but we are both fairly new to sex this being only the 3rd time this had happened so we still freak out every time we do the deed. Should we freak out even though we are being cautious and practicing safe sex? And given the circumstances should we have anything to worry about?