boyfriend changed his mind

So I'm pregnant with my first child right now but my boyfriend already has three (he's 11 years older than me). I told him in the beginning that I wanted to have a big family (at least 4). He used to be okay with that but now he's saying he doesn't want anymore kids. I just feel conflicted because for most relationships this would be a deal breaker. But I'm already pregnant. I want to have more kids. Now I'm wondering if I'll never get that opportunity because he changed his mind. I don't like the idea of leaving him at this point so that I can have that big family that I've dreamed of. Obviously we're very serious and in love but I'm just kind of upset about this. I mean that's a big deal to me. I don't know how to feel or what to do about it, if I should even do anything.