Possible infertility.

I'm in my early 20's and I'm wondering if infertility is a possibility for me. My husband and I haven't necessarily been trying for a baby but we haven't ever been careful either. After three years, nothing has happened. I have a history of super bad periods and my gyno has brushed them off and told me I was fine without examination. Any opinions?