jokingly call my boyfriend gay and it seems like it turns him on...

Okay so long story short: him and his friends have gay inside jokes. Well one time I made a joke and he got really worked up and immediately wanted to have rough sex. He acted irritated but laughed but like was like assertive sex (not forceful just dominant) so I tested it like 3 more times and the same result... 
Doesn't that strike anyone else as weird??? Like I made some pretty homosexual references regarding him and his friends for it to happen each time because after the first time it struck me as odd because he's not usually hardcore dominant like that...
Edit: I'm definitely wondering if he is bisexual. I asked him flat out the other day (I wouldn't care because I, myself, am bisexual) but I feel like I'm bothered by it so much because I feel like he's in denial/lying to me when he says he's straight. Time will tell right?