Sleeping around

I am currently sleeping with 2 different men we are in polygamous relationship. But I want to expand our relationship to 4-5 my boyfriends say NO but honestly what’s the difference between 2 & 5

Edit: I don’t think it’s messy, because what is 3 more men? My guys now have a good relationship & were open so what’s the problem besides “messy”