Should we worry?

My husband and I have been having unprotected sex for 11 months and are not pregnant. We have Not been "actively trying" in the sense that I track my periods but also don't use ovulation tests etc. we were hoping it would just happen when it happens. Should we worry?