Thinking About Having Another...

Soo, my husband and I have 2 kids, a 5 year old boy and almost 4 year old girl. They're as amazing as a 5 and 4 year old can be, but I really want another kid. I've always wanted a big family, my husband did too..at the beginning. But then he saw that it's not all sunshine and daisies after our first, and I had to convince him to give my our baby girl, which he loves her to death. But every time I bring up wanting another, which I don't bring it up often (it's like I've shamed his great ancestors when I do), his point of view is always "why do you want another when you have your boy and you girl? We've got the best of both worlds, we have our 'perfect' little family with our son and daughter, we don't need anymore kids". So I'm just like, alright I guess that's a no. Have any other mommies struggled with having this convo with their SO? And if so, how did y'all get through it?