Because i find girls attractive that means i have to have a threesome?

So this guy i been with for almost 2 years keeps asking or suggesting once and a while that he wants a threesome. I dont want to at all . Is it me being selfish ? Or how should i respond to that? He thinks just because i find girls attractive that i should want to do it. And because his last girl was not attracted to girls he never asked her.