Asking if someone is pregnant if you aren't sure

Do you ever ask when someone is due if you don't know them? I've never gotten asked this question in my street clothes but the work uniform I've gotten this past year has caused a lot of confusion. It looks like a typical maternity dress. I'm not stick skinny but I'm not fat by any means. Well I've gotten the question "when are you due" at least 7-9 times this year. When I say I'm not pregnant they always say "oh I'm so sorry that dress just looks like a maternity dress!" It's gotten to the point where I just go in the back and cry. I just lost 5 pounds and have been feeling so positive and I feel like it'll never be enough. I was asked twice today when I was due. I was always taught not to ever ask because you don't know if that person is struggling with their weight or if they have a tumor or if they JUST gave birth. You just don't ever ask especially if you aren't sure. I guess no one else got that message. Does anyone else ask? Or has anyone been asked and have it affect their self esteem? I'm just so upset. And my friends can say what they want but none of them have ever had to go through the embarrassment of having to act like its ok when it rips your feelings apart.