Social Pregnant Anxiety

So my husband and I decided to wait a bit after we married to have a baby. For the last 4 years I have been bombarded by the worst group of mothers pestering about when we will have kids, if ever ( that comment really hurt). Now that we are trying I am super excited, but want to keep it a secret so the "mean moms" will leave me alone which is depressing. . . Anyone else have or had a similar situation? Would it be totally rude to just ignore them? I don't handle stress well, sometimes confrontations bother me for days, can't imagine how to cope with raging hormones.