Do you take baby out for walks in the cold?

I live in Chicago and it'll be around 20-25 degrees tomorrow. I hardly go outside because my bf gets mad that baby will get sick since its been crappy weather. Baby is about to be 5 month on the 20th and I'm ready to jump off a roof or something. Help!! Is 20 degrees too cold? I don't want to go for a walk if it will do harm. I was planning to go to the store which is a 25 min walk (going) and another 25 min coming back.