My partner of 3 years and father of my kid gets really uncomfortable and weird if we talk about previous relationships.

He wasn't always like this, when we first started dating he randomly out of nowhere told me I was the 7th person he's had sex with. More recently when I brought that up, he said that's weird, doesn't sound like him, he "can't remember" how many, blahblah. When I pushed one day because I thought he was being super weird and sketchy, he got pissed off, counted off names and said "yeah I guess it's 7!"

I just don't see anything wrong with this conversation, it doesn't bother me to know who he's slept with and it doesn't bother me for him to know who I've slept with. Secrets aren't part of a good relationship. His reaction makes me think he's hiding something, and it just seems so stupid and shitty to keep secrets, and I go to really questionable thoughts as to what the secret "could be." but like why not just be open and honest? Am I wrong here?