Help!!!

Okay so I'm 16, I started having sex when i was 15 and so far I've been with 4 people so far and I just got in a new relationship and he wants to have sex. I do to but I don't want to add to the number of people I've been with. Have I been with to many people for my age? Should this stop me from have sex with my boyfriend?