Middle and last name opinion

So we are having a baby boy out of wedlock... we don't discuss getting married but we are together... we both have children from a previous . I agreed for baby to have his last name... but I would like to give our baby my last name as a middle name so my son and I also have a name connection to him. What's everyone's opinion.. he says absolutely not.. I say yes because I agreed to baby getting his last name.. I would love opinions as this has been a huge argument 😢