What is wrong with me?

I've been married to my husband for 4 years. A month before our wedding we started having problems with our sex life, him not wanting to have sex with me that much. This has now been going on for the last 4 years. Everything else in our marriage is amazing.

This last year has been the hardest on our marriage because I finally got sick of not feeling wanted or beautiful to him. He never tells me I'm beautiful, sexy, pretty or hot. The most nice thing he will say is I look nice but that's very rare. I ask him things that will make him more turned on and he says things like he finds it hot when I work out, or if I had more self confidence or when I wear short shorts or if I initiate sex. I do those things and nothing changes. If I initiate it he usually tells me no he doesn't want to.

Back to the hardest year of our marriage. We fought so much the first 4 months of this year. Than finally things started to look up. He actually seemed turned on by me and seemed to enjoy our sex. We had a lot of sex most of it was initiated by him. He said nice things about my looks and even walked by grabbing my ass on a few occasions saying I'm hot. Well for about the past 3 weeks or so again he seems uninterested in me and doesn't say anything nice about my appearance. We are still having a little more sex than we used to but he doesn't seem into it.

My husband has said all I care about is sex but I don't feel like that's true. It's just the only problem in our marriage is sex. My question is their something wrong with me that I care so much about our sex life? Sometimes I get mad if we don't have sex, which leads to an argument. I'm afraid to initiate sex because I don't feel sexy enough and when I have initiated it, he tells me no. This has me waiting around for him and most the time he just goes to sleep and I'm mad about that.

Should I just try and get over it since everything else in our marriage is good? Also I didn't start really getting mad at him about it until this year so me hounding him about it wasn't the issue.( Until maybe this year.) Any advice is welcomed and please let me know if I'm just overreacting and I need to get over it.