Is it normal to lose attraction when your partner stops having sex with you?

So, my fiance and I have been together 2 years. The first year of our relationship was great, we had sex regularly, always at least 3 times a week but usually more. Now this past year, its been getting less and less. We went from 3 times a week, to once a week, to once every two weeks to going almost a month with no sex. I have a very high sex drive, to the point where I could literally have sex multiple times a day. My fiance has always had a lower sex drive than me but its been fine, I was completely satisfied our first year together with just a few times a week. But now? I've been waiting 3 weeks currently to have sex with him and he hasnt shown any interest. We've talked about it so many times, I've asked him what I could do and he says he really likes when I initate and if I were to give him oral he would be able to get in the mood for me. So, I tried that and it worked a few times but after that he just started pushing me away and saying he wasn't in the mood. Its gotten to the point where I have been rejected so many times that I dont even want to try anymore. I love him, but as our sex life gets less and less I find myself having a really hard time being attracted to him physically. Even when he kisses me or touches me I find myself pulling away and I hate it, I can't stand losing him but something needs to change and it feels like he's not willing to take any responsibility. He's even said multiple times he would go talk to his doctor to see if there was anything that could be done and its all talk and no action (no pun intended lol). I'm just frustrated.