Is this worth leaving him over?

Me and my husband have been together for 13 years and have a son.. Last night we got into a fight because he never spends time with me. He is always working. And when he is home he is always sleeping or on his xbox or phone. He had some free time last night so I asked him if he wanted to watch a movie or play some xbox together and he says he's too tired but 5 mins later he gets up and goes to the track with his brother and our son. I told him I felt lonely even when he's home. And we got into an argument. I told him if he doesn't want to spend time with me then I'm not obligated to have sex with him and he said "then I'm not obligated to love you." That hurt and made me feel like this whole 13 years he has only been with me for sex. Then the next day he said he didn't mean it like that. He comes home on his lunch break and I ask him "so the only way you will love me is if I have sex with you?" And he says "no but sex is part of marriage and if you are not obligated to have sex anymore then I'm not obligated to love you" then walks out. I feel like that's what he means and should just be done with him.