boyfriend problems

my boyfriend says he loves me and wants to be only with me romantically and have a family and a future only with me but then says he sometimes wants to sleep with other people. I suggested we take a break but he doesn't really want to. I don't want either of us to be unhappy down the road. what do you guys think?