relationship advice im begging for help!

okay so here's the story I have been asking my hubby about his past n this is what I got. didn't lose his virginity till 19. has only watched porn 1 time since he was 19 now he is 40 years old.. been to a strip club one time. been with only 6 women including me n they were all long term relationships more than 2 years some up to 4-5 years. went 5 years faithful to a woman 2 of them years it got bad n was never sleeping w her n never watching porn or anything. she was right before me.. only lived with one woman never talked about marriage or children with any of these women n never even thought about buying a ring. never went down on anyone but me! never sexted or did anything crazy. sex in bed only with these other women. n says they only had sex about once a week besides for the first one which was more often than that. now all I think about is them sleeping together n it bothers me so badly. he has the past I could only dream of in a man so why am I so worried!!!???