Husband troubles during pregnancy

So my husband has been distant since weve been pregnant, we both planned for this pregnancy but its like he got cold feet when those two lines appeared. He doesnt touch me like he used to, his sex drive has gone down, and he doesnt interact or show interest in the baby or my belly. We talked about how I felt and he acknowledged that hes aware hes been distant but “doesnt know why” he says hes just got a lot on his mind and worrying about the future with a child in the picture now, he acts like his life is ruined and our child is now a burden. He says hes still attracted to me, and I brought up feelings of him cheating with all my insecurities ive been feeling. Well he later tells me a story of a woman they work with who is “not attractive” according to him but made a sexual joke towards her because they “treat her like a man” and she made one back at him after he tells me she “hits on him.” Telling me she sends pics of herself to another guy they work with and that its gross and she looks gross blah blah that was the point of the story that they “pass her around as a joke” amongst the guys. That really doesn’t help my feelings of insecurities and now im the bad guy and hes pissed at me because thats what I got mad about during the story. When my ex cheated on me with the girl he left me for, he also told me she was “gross, not attractive, a lesbian” everything and above and I had “nothing to worry about.” Well obviously I did so its hard to trust a word that comes out of their mouths sometimes. Why are men so ridiculous? I feel so alone, I cant even talk to the one Im married to😔