Relationship help...

So my boyfriend and I we're out at the bar with a few of his buddies when I got there my boyfriend and his buddies had already had a few drinks. We were all conversating and everything was going okay and then our sex lives all came up but we still kept other conversations going as we normally do and one of my boyfriends friends came up in the conversation because she comes and does some work for them every once in awhile and his friend asked me how I liked her and I said she was okay. She's kind of like me and my boyfriend goes don't you say it I know what you're thinking to his friend that works with him so his friend goes yeah she's okay I think she's a little bit weird and then my boyfriend's like Oh I thought you were going to say something like we should have a threesome or something and then my boyfriend kept going on about it I don't know how to feel about that I don't know if it was just him jokingly saying it or that it might be something he wants for real. I've already had a few of my ex-boyfriend's leave me for their best friends I don't want that happening again. What's your guy's opinion on that. Am I just overreacting or is it something I should talk to him about?