Feeling hopeless

My fiancé and I have decided we wanted to start trying for a baby but he doesn't have a big sex drive and sometimes it's hard for him to stay hard during the act and very rare that he even finishes. I am worried that I will never have children due to this. He has been to the doctors and they say there isn't anything wrong. He was given samples of Viagra and it didn't really work. I don't know what to do, sometimes he is a champ but more often then not its a dud.