Should I have sex with him?

I made plans with the guy I like for him to take my virginity. He's really nice and mature, but there's one tiny 'problem'. He's a polygamist, as in, has a lot of sex with a lot of different girls, so I will be just another girl for him, although he does realize it will be my first time and wants to make sure I'm comfortable and happy with everything... Will I regret this? Shouldn't I do it with somebody who also likes me back? Am I moving too fast?