Afraid of Sex

So, my partner and I fight every now and then as do most. But most of it stems from one situation that I seem to not be able to get past. About a year ago I tried to make moves on him because I was horny, naturally. He wasn't picking up on it, so I started touching him and made my way down to the obvious location. He got really mad at me. He told me that that's not how you do that, and made me feel like a predator. So, naturally I stopped initiating sex for a while, because I want to be mindful and respectful. I still do to an extent. But all of our arguments since seem to be centralized around that occurrence whether the argument started out that way or not. When he gets defensive, he uses low blows and says things "he doesn't mean", like "well you just lay there" or "you don't try anymore". I understand where he's coming from but he's scared me into to being totally submissive. I don't want to do anything wrong anymore. I also have a hard time being on my knees for a long period of time and he knows that. He says he doesn't mean these things but he picks the things that hurt the most. I know these arguments may not need to happen and I am sometimes, more than not, the person who starts them. Isn't there some truth to everything? Especially when you're picking hurtful things that are relevant? He seems to think otherwise. I love him, a lot, and we have a lot of good things going on outside of this but sex is such a chore now because I don't know what to do. He wants me to be confident but I cant. He also doesn't take his time during sex anymore and just rushes into things. For me personally, I don't always need penetration even, I just need closeness, but he doesn't understand that.

I think about this a lot and some days are hard, and I feel the need to talk about it with him. But I can't without it escalating or him saying "he's trying". It doesn't seem like he is. He says I need to be patient with him and I demand the same, but it's been a year. What do I do? Any advice would be so appreciated. My self esteem is down pretty low because of these problems.