Why is sex so painful?

Ever since I started having sex it's always been painful. At first I was like yeah ok I get it the first few times will be like this, which was true cuz the third or fourth time didn't hurt, but everytime after that kills and rips me up a little bit. I just have no idea why it hurts so much, we use lube and I do my best to relax when it's time to put it in and we go slow putting it in. Once it's in it doesn't most of the pain goes away and starts feeling good, but the pain kinda takes me out of the mood. I just don't know what to do I feel like I've done everything, why is this happening? Any help will appreciated thank you☺