Boyfriend says dumb things sometimes

Sometimes he’ll say I’m “half gay.” [im bisexual/pansexual] and then follow it up with implying that I’m straight right now because we’re together?

It really frustrated me last night and we got into a semi-argument about it. I told him, I’m not straight, I’m not gay, I’m bisexual.

I guess I’m just frustrated that he thinks he “won” by turning me straight.