Family and trying

My husband and I are trying to conceive, but we haven't told anyone on either side of our family. We are really close to his family and mine. I feel that when we do get pregnant everyone will be upset that we didn't mention we are trying. I am I the only one that has this issue and what should we do?