Emotional Manipulation

My husband told me that he was unhappy but would never leave me because I am emotionally abusing him by manipulation and that he could never leave me because of it. That I would have to leave him. Then he said that he would “fake it” for the rest of our lives because he doesn’t want our son to have two separate households. That’s he’s tired of my abuse?.... I have literally never abused him. He started saying things like this after he got physical with me (not hitting or anything but grabbing), and then I said that I was leaving because I didn’t want our son to grow up in this kind of environment. But I ended up staying because he started crying and we talked it out. I don’t even know how to respond to this?