Is it wrong to hang out with my brother’s ex?

My brother dated this girl for a year and a half which is his longest relationship. After they ended, he started dating another girl who I’m not exactly fond of but they only lasted 5 months. My family and I have a relationship with the first girl, who is such an amazing person, inside and out. Even my mom still talks to her. However, my brother told me that he isn’t comfortable with the idea only because he feels like she’s always gonna be the girl that we’ll compare whoever he’s dating with, which is untrue! But he feels that way. I love my brother but I also don’t wanna lose this friendship. When we hang out, we don’t even bring him up even though we talk about everything. Idk what to do.