I'm about to give up :(

Ashley
My husband and I have been trying to conceive for a year and a half now. I feel like nothing we do helps. 
I have tried to get him to go to the doctor with me, but he always says that they will blame him. 
I'm not sure what to do. 
I feel at times it is him, because he rarely last long and sometimes doesn't have the ability to keep it up. Does that mean something is wrong? Or am I just worrying to much?