I feel lost

My husband usually respects me but last night he said something so hurtful. He was first being sweet saying every morning he thanks god I am in in his life and our daughter that he feels content in life. I thought it was sweet I feel the same. But then he went to zero to sex really quick Im 18 weeks pregnant and not feeling frisky at all. He got upset I guess since I didn't want to. He said come on it's been two weeks (to him it's a long time) but I don't like to keep track of sex like its a chore I like for it to be a mutual feeling of the moment not because of we have to do it. Then he said don't say anything when I cheat on you. That's when he hurt me. He said that once joking but I brushed it off this time he was serious and upset.  We have never had infidelity issues. In his family it's very common the men do this to their wives and he's always said I wouldn't do that to you because I saw my dad do that to my mom for years. He knows cheating is unforgivable to me and I wouldn't want to stay in a marriage for my children I don't believe it's healthy for them to see an unhappy relationship like he did when he was growing up. I just feel sad and lost and now I'm thinking is it possible that it's really crossed his mind or has he done it already. I feel stuck I haven't said anything not even in anger because i was pissed at first but idk what to do or say to him anymore. Thank you for reading this!!