Wanting another baby

So my bf and I have been together almost 2 years. I’m 28 and have 2 kids from a previous marriage, a 4 yr old and a 2 yr old, and he is 31 with 2 older kids from a previous marriage, a 14 yr old and an 11 yr old ( he was a teen dad). He and I both agreed that we want 1 more child we just haven’t discussed when. We just moved in together in February so I know we need to wait a while before trying…. But I have baby fever SO BAD it hurts! I thought about bringing it up to him because I don’t know what he’ll say, and the thought of him saying no makes me kind of sad… should I push through and wait a while or should I just ask him out right when he would want another baby?