Husband doesn’t approve of sperm doner

me and my husband have been trying for our first child and it hasn’t been going great so we decided to get tested and we have just found out my husband is unable to have kids of his own so I asked him if we could go thru sperm to conceive and he said no as he says the baby will only be mine and not his and he will feel like a step dad which I don’t get as he stated he’s open to adoption but doesn’t want another man’s sperm in my body I’m currently 23 and so is he I would and I feel I should divorce him as this isn’t how I see my life iv always wanted to carry my own child and he knows this and I feel I need to divorce home or I’m gonna keep putting him and his opinions first and never what I want is that wrong