When should you stop and give up?

When is it time to give up and sign the divorce papers so we're both happy? My husband doesn't treat me like a wife, I feel more like a freaking maid. A burden, I can't do anything right and he just complains about if I touched his stuff. (Moved his papers for work etc.) he's complimented me twice since we've been married. 3 years we've been married. He says he doesn't care what I do, I work 6 days a week 8-10 hours a day and the one day I have off he wants to go to his mom and dads house. Which were literally over there every damn day. I just want a day to ourselves! It's like he's scared to be alone with me! Most the time he won't kiss me on the lips, never holds my hand. Rarely gives me a hug. All I wanted in a relationship was to feel loved and I sure as hell don't. So what should I do? I'm lost and I love him but when is it time to just let go? Please don't bash on my husband I just need advice. And yes we've talked about it multiple times