Is my fertility changing?

I was on birth control before I got married simply to make sure I wouldn't have my period on my honeymoon. My husband and I decided to start trying to get pregnant right away, so after the honeymoon, I quit taking the pills. I had my period right away, and got pregnant the day after it stopped (according to the doctor's date of conception). Now, my husband and I were each other's first time having sex at all, and he was having performance anxiety so he could hardly get it up, but I still got pregnant that fast. Now we are trying for baby #2, and thought it would happen just as fast, but we've been trying for 6 months now with no luck. Could our fertility have changed that quickly?