What if I’m not adequate enough??

So my relationship with this guy is kinda complicated, it goes back to 2 years ago and would take me literally hours to explain the whole story of me and him so I’m just gunna get to where we are right now. I love him and he tells me he loves me and we’re literally best friends. We’ve had the talk about “us” and we do wanna be together but he just got out of a relationship about a little over a month ago and yesterday our conversation was picking up and next think I know we’re flirting and talking about sex. This isn’t something new or anything, we’re always flirting but we were talking about us having sex and he said he wants to and I asked him what’s stopping him because I want to too and he says because his ex .... my draw dropped and I got so upset. He said he can’t have sex with me because he would expect it to be like his ex. I basically just said I understand and blah blah blah, we changed the topic a few minutes after that but I don’t know how to feel about him saying this. Like I understand because he’s only ever been with her (she was his first and only) but does this mean he’ll never have sex with me? Or when we have sex will he be thinking of her? So what are your thoughts about this? Should I be worried? What if we have sec and I don’t compare to his ex? What if she was better at it then I am and sex with me makes him want to go to her? Please help??