He cannot wear a condom...

My new boyfriend cannot put on a condom because he is never hard enough or as soon as we open one, he loses his hardness. I am not on a pill and don't plan to be for medical reasons, plus we are rather new and I don't feel comfortable taking any risk, being pregnancy or std. Does it happen often to guys? Does seeing a doctor help? He does not seem too open to speak about it as I was trying to start the conversation. I am getting bit frustrated, and I don't want to hurt his feelings. Any advice?? Thanks