Is it weird to hang out solo with your older sibling and their fuck buddy?

I’m always asked (she extends the invitation through my brother) if I want to hang out with them. I know my brother wants me to make new friends (don’t have many), but I find it weird to always hang out with them and make accommodations so they can do it alone later. she seems nice and friendly, but I don’t want to hang out as a third wheel. I just feel weird about it