My husband and I are arguing about our future baby’s name

I’m pregnant and we haven’t found out the sex yet but if it’s a boy my husband said he wants to pass down his moms maiden name to the boy just like he has in his own middle name. Both my husbands brother and sister have had boys of their own and included their moms maiden name in their children middle names aswell but I hate the name. Is it wrong of me to fight my husband on this? And because 4 other ppl in the family will have the name if we do pass it down I think that’s weird. The maiden name is McDuffie to make it worse. My husband was bullied in school for it.