husband talking about sex w other women?

so theres this big lady, around her early 50s. she lives above my husband and i in our apartment. she always makes banging noises, like as if shes banging her feet constantly on the floor, and her dog always runs around. my husband doesnt like it and was bad mouthing her, he was also drinking. he then said he wonders how hard it would be like to stick his dick inside of her. he said im skinnier and its already hard for him, so it would be much harder to stick it in her since shes so big. i was so mad, upset, and disgusted. he told me to not make it a problem or else things are gonna get worse between us. then later lying that he never meant it in a sexual way like WTF!? he went to microwaved the left overs for him to eat and he said he bet it taste even better than her. even though he was drinking, he was still sane, and still remembers everything. he wasnt even drunk at all. this makes me want to leave him. should i? and wont u be mad too if it were you?