Am I justified in leaving?

I don't want to leave him. I'm head over heels. We have a 5 year old son and long story, but we just started a relationship 9 months ago, been living together 18 months. Things are fantastic, until we start talking about the future and he hasn't said I love you (I've said it a few times, he just hugs me or pats my shoulder). Honestly all I really want from life is him our son, our children to have stability, and more children. Its been my dream since I was 8 to have children. Not just me but also for our son. If I got pregnant tommorroe, he would be 6 years older than a sibling. He's not interested (or seems uninterested), in marriage, or getting a house, or having more kids. Our son is very happy and so am I most of the time. Its just the quiet moments where I start feeling resentful of him for not wanting the same things I want right now. He can't or won't even give me a good reason.

I could live with him not loving me, of course it would hurt, but eventually I'd move on. I feel like he's holding me back. I feel like he's cowardly for not committing. I'm planning on just living in the moment for another year, but after that? I don't know what the right thing is. I watched my mom give up what she wanted from life over and over for a man. I don't want to do that.