How would you handle this?

My fiancée and I are finding out the gender in 4 weeks. We've talked a lot about names since day one. 
We decided on Koi for a boy, and Zen for a girl (like it or not that's what we like I'm not here for to be told you don't like it I'm here for different advice) 
We know the names are different, so we want to give our kid a "proper" middle name, something more "professional" to go by if he/she prefers. 
He has always told me he doesn't like the idea of naming our kid after any family. 
This week he dropped it on me that instead of Zen for a girl he wants to use the name Flora, after his grandmother who passed away almost a year ago. Before we were considering it for the middle name but I really don't like it because I also don't want to name the kid after any one. They were close his grandmother raised him, and I think it's sweet but even she HATED the name Flora and went by something else her whole life and it's just not what we agreed on in the beginning and I'm willing to compromise so I asked him if we did Zen for a middle name instead if he would be offended if I called the her Zen because I feel like it would fit better (I just simply can't picture myself calling my child Flora) and he said YES he would be offended because it would just be me "not getting what i wanted" 
I don't feel that way, I'm just trying to make us both happy even when I really don't like the name Flora. I just don't t know what to do or say to that. I don't want to hurt any ones feelings and want every one to be happy.