Taking Newborn out

I've taken my baby who is now 2 weeks old out of the house a couple of times to Dr. Appointments to lunch and to the store but people keep telling me not to that I should be home in bed rest and baby shouldn't be out .. What have you moms done since giving birth ??? Or what's your opinion