Problems with erection

Okay so, me and my boyfriend had sex for the first time. Everything was going great we were both turned on he was hard but... as soon as he put the condom on his penis dropped. He put it inside and it was hard for a few seconds but then it started to go soft. We thought that it was just the nerves because it was our first time and we were both virgin. So we tried again the next day. The same thing happened and we couldn't have sex. As soon as he took the condom off he instantly went hard again. He has no problem keeping his erection without a condom though. Any reasons as to why is this happening?