Trust Issues In Sex

I've recently got with a guy who I completely adore. He's had sex and I haven't due to having trust issues because of a previous boyfriend who hurt me during sex (basically he forced his penis in dry,and it hurt so bad!). I want to have sex with my new boyfriend so bad but I'm finding it difficult to fully trust in that area. I've spoke to him about it and he says he's not bothered but I know he is because he makes little jokes about it to lighten the situation. I need help to try make myself comfortable with the situation. I'm scared that he will leave me because I can't give him sex. I know it's silly but I constantly have those thoughts.