Fiancé won't let go of my past!

Okay so I'm 22 and so is my fiancé. In my highschool days I was kind of on the wild side and I did things that I'm not proud of. I don't know what was wrong with me I was really rebelious. I'm honestly ashamed of my past when I sit there and dwell on it, and when we first started dating he would tell me about all of the things that he heard about me that really weren't true and I would tell him that what he heard isn't true. He always says "how come everyone else lies about you". It hurts because there are a lot of things I'm really really not proud of but the stuff he tells me he "heard" about me really are not true. He really makes me feel really bad about myself when he brings up my past and it honestly make me feels less of a person even though what he heard isn't true. He gets mad every time he even thinks about one of ex boyfriends or anything. I wish I could change the past but I can't. I just wish he would let my past be my past....