My boyfriend always makes comments about me being lesbian?

First off, I am not lesbian. I experimented like sophomore year of high school, but I like guys. There’s nothing wrong with being lesbian, a majority of my friends are lesbian or bi. But for some reason my boyfriend always says shit about me being lesbian, like oh that’s why you always hang out with taylor, or you’re just using me as a coverup etc. And it gets kind of annoying? Because I’m not, and I try joking back with him but he always says some shit, and it gets kind of old sometimes. He’s also said shit that if I hooked up with a girl, he wouldn’t consider that cheating, and he would be fine with it. I feel like he has some sexual fantasy and he’s trying to push me to fulfill it? It literally happens every day, and if not, at least every other day. Am I making too big of a deal out of this?