How do you guys feel about the past and trust

I just started dating someone and we have known each other for about a year. When we were talking in the past I was also talking to another guy. He asked me when I broke up with my ex and I didn't know the date. I know it's normal to be cautious but he lost trust with me. but he was doing the same thing talking to another girl and now we both decided to comit to each other but he won't let go of my past... What should I do?