Confused

My SO (not the bio father of my one week old) always says he loves my son more than he loves me.. Then he says, 'just kidding' 
But I don't think he's joking. He also isn't affectionate with me anymore. Like at all. I don't think he loves me anymore at all actually. We fought tonight and he isn't even trying to fight for the relationship to work. All he really said was that even if I decide to hate him, that he would like to see my son tomorrow. Tips? Blah