I know we are going to have trouble conceiving, but can't tell DH

When my husband and I started dating, we initially used condoms, but when we became exclusive we stopped.  I was taking depo at the time, but had lost track of when shot due, and was out of town for work for 3 weeks.  We were both under impression that we were protected against pregnancy, and when I realized, I very poorly decided not to tell him.  The first year together we had sex daly, now it's more sporadic, but on average 3-4 times a week.  I've never gotten pregnant.