getting out with baby

This is mostly a rant about stranger comments. My baby is three weeks old, and I've been going out with her. I went to the doctor, the grocery store, and with my husband along, to Barnes and Noble. Each of these outings only lasted perhaps two hours because my daughter is breastfed. She gets hungry and ready for a nap by then. However, each time I've gone out people ask her age, and when I say "three weeks" they act as if I'm a nutcase for taking her out in public this early. They usually say something like "I can't believe you're getting out." It makes me feel like I'm doing something bad or abnormal.  I get so bored at home, so it's a nice change of pace, and I feel good and my baby's fine in her carrier. I don't understand why people act so shocked that I'm out.