When did you first go out with newborn?

My baby is 3 weeks old and I’m going stir crazy sitting in this house. I would like to go out somewhere with the baby and husband but idk where to go and I have a fear of him being fussy and crying the whole time. When did you first go out to eat or to the store with baby and do you have any tips?