No Sex Life

Need some advice. How can I get my husband to touch me more? We have been married 3 yrs and dated 4 years before that. He’s never really been touchy. We rarely have sex, but when we do it is amazing and he always says “we should do this more often” but he literally doesn’t touch me, unless I touch him first. I get little kiss in the morning when he leaves for work and a little welcome home kiss. We have talked about why he doesn’t touch me, because I feel like he’s just not attracted to me, but he just says he doesn’t know if he’s “allowed” to. I tell him I crave his touch and want him to violate me sometimes and if I don’t want to be touched I would just tell him. It drives me nuts. It has been like this our whole relationship. He is a very respectful man and has so much respect for women, but I am his wife and just want to know he wants me. I bring this up to him every couple of months and nothing changes. I try to send him little sexy texts and I usually just get an emoji back. He says he’s paranoid about “talking dirty” over texts. It just doesn’t seem normal to me. He mentioned his last relationship the girl was very “weird” about their sexual relationship, but won’t tell me anymore than that. It makes me feel like I can’t initiate anything because I’m not sure if HE is ok with it. Not sure what to do, it’s really damaging to our relationship. Appreciate any advice.