Taking baby out in public

When did you take your babies out in public when they were born? My son is a premie born at 35 weeks so he came before I had my baby shower. He’ll be a week old at the party and I’m getting some people telling me I shouldn’t bring him. His pediatrician appointment is today so I’ll be getting his doctors opinion as well.

ETA: I planned to baby wear the whole time and I’m EBF so I’d have to cancel it if his dr advises against him going.