can anyone help put my mind at ease???

So my fiancé was in a 5 year serious relationship with his ex partner(whom he was engaged to) I have found out since that when they were together, she was very similar to myself & could not have any contraception as it did not settle in her body, obviously being with someone for 5 years I am well aware that they will have had unprotected sex numerous of times but now I am wondering since myself & my partner have been TTC for around 5 months now that is has occurred to me, is it a coincince that him & I are struggling to conceive & that he was with someone for that long & she never conceived either?? 
I have since found out also that his ex is now pregnant with another's baby. Can anyone relate or offer some advice? I know this post may seem strange but I can't help but overthink things...