TTC and different sex drives

So I'm assuming most of us are trying to conceive but am I the only one who feels that I'm never going to get pregnant since my husband has no sex drive :/. I want my husband all the time, he could call me during any time off the day and i would drop everything to make love to him or even just have a quickie while I'm not ovulating. My husband on the other hand seems to never want sex. When we do have sex he seems to enjoy it but why do i always feel i have to practically "rape" him to fulfill my sexual needs? Does anyone else have issues with this? What have you tried? I have talked to him about this before and he says he's attracted to me and always wants me but he never chases me :/ I feel lost.