Sexual compatibility

Okay so I met this guy. And I like him a decent amount (it's early) but... two things. One, he is not the largest. And two, his dick curves downward. (Also he's uncircumcised, which is new to me) so because of this, the sex doesn't really feel that great. He's also pretty inexperienced which doesn't help. And it barely stays in, and it's so hard to put it in at first unless I'm on top. I don't want the fact that our sex is meh to ruin things! He's so nice but I just can't deal. Am I shallow for thinking this?!