Am I wrong for letting this bother me?

Recently I've been bothered by my husbands past, particularly a certain relationship he was in. About 6 years ago he was in a relationship for about 2-3 years. During this relationship, He said he wasn't very happy but he stayed with her because he didn't want to be alone anymore. He told me she was a drug addict and was always in trouble. This girl told him that she wanted to have a baby, and him not wanting to be alone, he decided to try and give her what she wanted. Anyways, During this relationship they tried many times (I hate thinking about that) and she ended up having 3 miscarriages. He didn't tell me that willingly, I had to harp on it because I suspected something major had happened in the relationahip. He said that he didn't think he needed to tell me, and he never planned on telling me because they were never born, so it doesn't matter. He also might potentially have a daughter with her as well but he isn't 100% sure if it's his or not, and he's never been in her life because the mother refuses to let him, and she herself says it isn't his. But people have told him that she looks a lot like him.  This has been bothering me lately, especially more so lately because I'm possibly pregnant with our baby. Am I wrong for letting his past bother me? 

Vote below to see results!