Conversation with husband

So last night, I was very in the mood. The problem is, my husband only likes to have sex once a month. He's 28, I'm 23. I feel like we should have a much better sex life than that, especially when we are TTC. I asked him last night, is there a reason that we don't have sex often? His reply was kind of shocking. He told me I was over-weight and he didn't find that attractive and finds me more attractive with the way I dress and do my makeup than when I'm naked. He said we would have sex more if I went to the gym 7 days a week (no joke). I have always been comfortable with myself and in my own skin. I'm 5' 8" and 160 pounds. I was offended. I love him but that really hurt me. I mean, I don't know what to say. That's the first time in our two and a half year marriage that I went to bed truly angry. Has any other woman been told this by their SO? Or is my husband just a douche bag?