taking out newborn...

I'm a first time mom, and my baby girl was born on the 21st. I had asked the doctor if it's ok to take the baby to 2 places for thanksgiving (my family's and my boyfriends family's houses) she said it's best to just stay home because she is so new and her immune system is still building and plus because it's cold here. (We live in Chicago) Well my boyfriend wants to go Black Friday shopping (I have to take him because his license is suspended at the moment) and we don't have a baby sitter so we'd have to bring the baby. Idk if I feel comfortable because of what my doctor told me. Do you guys think it's ok, or should I put my foot down and tell him no???