Do I have a problem!?

I recently got remarried, very happily married for 5 months now. Let me just start with my husband and I both were in relationships where our spouses had cheated.
Thankfully we found each other. On another note, when my husband and I first got together he was in Georgia and I was in California working temporarily, when we finally saw each other after spending 3 months apart we had sex constantly. We still lived apart up until we were married 8 months later. We had sex 4-8 times a day, now I find myself crying in the bathroom because we only have sex maybe 1-3 times a week. 
My husband and I are very open with communication, we have a amazing relationship but he seems to think I only want a physical relationship. 
I love him, and I'm overly attracted to him to the point where I want him all the time. I take it personally when he doesn't try to have sex with me daily. He says he wants to lay with me and have more than just a sexual relationship. 
I can see what he means but what guy doesn't want sex all the time right? 
Help please