Should I have spoken up?

I have a question? Had an hvac man come to my home to give my husband and I an estimate. The man said a few inappropriate things such as the woman is always right, happy wife happy life “my husband rebutted with not if you’re miserable” then a little later the man kept ranting on how girls like me normally marry doctors and not guys like them. I was uncomfortable with the things he was saying but my husband did not say anything to really put the man in his place. If this were a woman I definitely would have confronted her but boot was I man I believed my husband should have. Unfortunately he blew up on me for not defending us when the man disrespected him and myself with his comments. Should I have spoken up just want thoughts