Spermicide?

We use Gynol spermicide but not all the time. We are not against having kids right now so no biggie if we get pregnant. It seems like so many people get pregnant even with using birth control methods that are very effective. Should I be worried that I haven’t gotten pregnant yet? We talked about it and we are both slightly worried that something could be wrong with one of us as far as fertility goes but we know a doctor wouldn’t take us seriously until we actually were trying for kids. I suppose I’m just worried about in the next year or two when we start actively trying to get pregnant.