sex first time question

So I am very scared to have sex but been with my bf for 2 years and know he's the one. Just the idea of him on me scares me. He can only finger me with one finger he's tried two and it hurts so much. we've talked about sex but can you tell me if this is right or wrong if I can't do two fingers I definitely can't do the real thing with him right??? It's not like I'm not wet and we've tried lube and everything. Is there something I'm missing?