Baby #3 or no baby

Ok me and my husband are 37yrs old we have 2 boys 12yrs old and 1yrs old. Since we've had our last lil boy I had to become a stay at home mom which I love, now I want another baby (Really want a lil girl) our parents feel that we don't need any more kids and my husband keeps talking about we need to be more ready financially since what happen with my job after 10.5yrs. But I'm getting older and really upset, so I'm kinda feeling like I just don't want another baby, but I can't stop thinking about what if I get pregnant and it's my lil girl. I don't know if I should let it go or keep trying to push the issue.