Is the age gap too big?

I recently turned 25 and I all I want is to focus on a career, find a house and settle. I want to start a family soon. I constantly think about having a child and raising a mini us. My boyfriend on the other hand won't be 21 until July and he gets quiet and pushes the subject away. In a way, I understand because I didn't want to settle at that age either. But at the same time, I just feel like he doesn't want to compromise or even talk about the subject at all. I truly worry that we're never going to be at the same place when it comes to these things. I love him and only see myself waking up to him every morning. But I also hate the idea of either of us wasting our time if we don't have the same goals.