Husband not coming to sister's wedding because he doesn't like my family and they don't like him

My husband said he's not coming to my sister's wedding. My family doesn't like my husband and the family eventually became mutual to the point my husband said he's done and he will not be around my parents, especially my dad or my siblings or anyone. There is so much bad blood and I just wish everyone could get along. Since my husband is not coming that means I have to choose between leaving my 6 month old with him or taking a baby to a wedding by myself.

Edit: My family doesn't like him because they feel he stole me away from my cheating ex who they did like. And honestly while he cheated on me he was just a shitty person to date. He was a nice person so I understand why my family liked him but he still sucked. As for my baby. I just feel like I would be thinking about the baby too much.