TTC for 3 yrs-Help

I'm 37 my husband is 40.  One of the  things in common we had while dating was our desire to have children.  I just don't understand why it hasn't happened.  God know my heart and my husband's heart.  Should I just give up?  Please help me ladies.  Sometime I just feel alone, hurt and embarrassed.