Boyfriend wants another baby.

My boyfriend and I have been on and off for 5 years. He has one daughter from a previous relationship and we have a 2 year old son together. We meet when we were very young. I was 18 and he was 21 so granted we've had a lot of issues. We even broke up for a year after our son was born. About 3 months ago we've recently reconnected and things are better than ever, he's grown into a very mature man and has been extremely helpful and attentive to me and our son. We've been taking things slow but he's expressed that he wants to have another baby with me. I would love to have another child but I'm hesitant because he and I have a lot of trust issues. Plus my family and friends would judge me for it. (They kind of distrust him for his past mistakes). I love him and would love to have another baby I just don't know what I should do.