Went through my phone!

So my fiancé and i are open with our phones, we don't have anything to hide. So he went through my phone a few weeks ago, and never said anything about it until today. He's the first guy I've ever officially dated, but I talked to guys prior to him. But most of them ever got serious. He only knew of two of the guys I talked to, because they were the only two that got serious. One was off and on for 5 years, and his best friend was about two years in between there somewhere (but it didn't affect their friendship at all, it was all pretty chill). But the ones that only lasted a month or less, I never mentioned. I didn't see the point. Well he found old messages (from 2015) between me and one of the guys and was LIVID that I never mentioned him. But we literally talked for a week and it wasn't working so we stopped. He blocked every guy who's messages he found off all my social media, even guys who were only ever just friends. I mean, I don't really talk to any of them now, but it annoys me. He's still friends with his ex girlfriends (many of them) and even his ex wife. Even when his ex wife was trying to get him back, I wanted him to block her and he wouldn't. Do I have the right to be annoyed???