Has Anyone Had The Same Problem

I was 14 my Bf was 19 when we got together. Now we been together for 5 years and have one Child. I Stared having sex with him at 15 at first we had protected sex. After couple of months we will not protect our selfs when having sex. The first time he came inside i was worry i might of end pregnant but i didnt. We will have unprotected sex and i woulnt come out pregnant. Which i was very disappointed. I thought i was never going to have a baby.
I turn 16 and i wouldn't still get pregnant. Untill i turn 17 (4 month to turn 18) i got pregnant. Now im 20 and i want to have another child. My worry its that i might have trouble again to get pregnant. It took me almost 3 years to get pregnant from my first baby. So im scared to try and this is stressing me out!😩😞. 
Has anyone had this problem before? Share your story please