Bring baby into the mix 😞😞 would love some aadvice

So my story so far. Ive been married almost 10years, and have been with my bf on and off for about 6years. My husband and i have been trying for baby #4 for a while without much luck. Last night my bf asked if we were going to use protection (as im ovulating in the next few days) or if we want our own baby! We never really spoke about our own baby before, and i was excited about the prospect of it all as im so in love with him more then ever. But with the trouble im having with y husband i just dont know how that would work. Should i talk with both at the same time or should i just talk my my bf and figure out if we were to do this how we would go about it all? Has anyone ever been in the same or similar situation? Any advice would be great