When getting back with an ex...

Do you think it's important to know who they've slept with during the time you weren't together? Me and My ex who I'm pregnant by have recently gotten back together. We were together 4 years before we broke up and we were each other's first everything. Anyway everything is good, we have finally got the baby we always wanted and decided it's worth giving it another go. He said today do you think we should tell each other everything that's happened since we haven't been together so we know everything and people can't try and cause trouble. He said this because there were a lot of people trying to come between us before it was crazy. Anyway I said no because what happened when we wasn't together doesn't matter and is irrelevant in our relationship now. Do you think it's best to know or leave it?