Conversation about a possible baby #4

My husband and I have been discussing about having baby#4. He has said he would love to try for a girl, (I have 2 kids from a previous relationship,1 boy, 1 girl) and we have a 1 1/2 y.o boy. I was still undecided of i want another. I do but I don't, so what i thought. It wasn't until tonight that he said that he thought it would be best if we didn't have anymore. Me being on fence, I understood his decision, but it also sucked that I have been of birth control for months now, trying to have baby #4, and now he full on decided that is not a good idea. I thought I would be okay with not having one of we decided before I came off birth control, but I feel invested and I'm having a hard time. I spoke with my husband about it and he is on board if I am currently pregnant but if not, he wants me to go back on birth control.

I don't exactly know what I'm looking for here but maybe just needing to vent. Thank you everyone for listening to my, not so important issue as some others, but I needed to get this out.