Finding each other after both suffering miscarriages

Me and my boyfriend are in a new relationship with each other. Both of us have suffered devastating miscarriages, with our previous relationships. We love each other very much, and plan to get married. I know I should be grateful and excited, but I lay awake at night, wondering and worrying, about our ability to have a family. We both would be great parents. I'm feeling like a damaged woman. Is it even fair to marry him, denying him the right to be a father? What if I never get pregnant again? I feel like the right thing to do, is let him out of the relationship, so that he has a chance with someone else. I know that I should probably talk to him, but he will just say he loves me, and we were already complete people, long before the situations and relationship. I'm just feeling broken now. I want to end it now, before we fall further in love. Trying to cause the least amount of pain. I just don't know.