Confused....

My husband and I are not actively trying to get pregnant but he has came inside me a couple of times in the last year and I never get pregnant. I have asked him if he is not telling me something or if there is a problem. He is sure that he is able to have kids. I have no health problems and I have check ups regularly. I secretly hope to get pregnant but it hasn't happened yet. I am confused because my friends are all getting pregnant so easily. What do you guys think?