Advice please

So whenever my SO and I start fooling around and he goes down or fingers, I don't ever orgasm, not even when I treat myself. The first time we had sex was the first time ever for me and I didn't orgasm then. What's wrong with me? Is it just that we are both inexperienced and don't know what we are doing? Or am I just worrying too much?