I want to have a baby but my boyfriend isn’t ready

My boyfriend and I have been together for 3 years. When we first met he had a 1 almost 2 year old. We discussed on our second date goals. He came out and said he said he wouldn’t ever get married again or have another child, but then realized it wasn’t fair to the other person and he would definitely do it all over again. We moved in together a year and a half after dating and we have his daughter who is almost 5 half of the time. I bought my house before I met him so he moved in with me. We purchased land together, and planned on building a house next year. I decided I wanted to wait a couple of years to build because I wanted to save more money and I already have a nice house. He was okay with that since he didn’t like the price. I noticed him acting different the last month or two. Every now and then I want to talk about the possibility of trying to conceive in the near future. The first excuse was he didn’t want to move with a baby, so I made sure to get the ball rolling on building. After we decided to wait I asked him again. He beat around the bush for days and then he flat out told me he just isn’t ready. I’m almost 30 and he is almost 32. I know people say you have time, but I don’t want to be in my 30’s when I give birth. I only want one child. His first marriage was a mistake he says all of the time. I feel that he is taking out what happened to him before on me. I am financially stable, I have a great supportive family, but he refuses to compromise. He asks me if I can understand how scared he is and yes I do understand. But I asked him if he could understand that I wanted to be a mother and he can’t understand. I’ve helped him raise his child the last 3 years and a pretty important part of her life being a baby transitioning into a toddler. I fixed her room up nice for her, I take care of her. Am I being selfish for wanting a baby? Isn’t that normal for a 29 year old woman to want? A family?