My boyfriend wants a threesome

My boyfriend just brought up the idea of having a threesome. I told him that I wouldn’t be able to and he just pulled the “I thought you weren’t the jealous type” card. I am not the jealous type and I’ve always been firm in my beliefs that if I feel like I ever have to compete with someone else then I won’t be in that relationship, but when it comes to sex it’s a while other ball game. It would kill me to see him with another girl even if it’s just physical. Am I wrong? How do I deal with this?