Baby refusing to eat...

So my baby is 5 months old and since day one have breast and bottle fed because he was in the NICU. Well He's never given me issues with eating from a bottle ( we struggle with breastfeeding ) but today I gave him his first taste of banana baby food and since then I've had two situations where he's refusing to take from a bottle. He just won't eat. I've never experienced this before... he took throughout the day..I always try breastfeeding first,then bottle and that's been the routine since the get go but not today....I'm starting to worry as he's crying like he wants to eat but is flat out refusing...it's 3 am and he usually wakes around 4 for a feeding but woke up at 2 and is crying and crying but turning his nose away. Could this be from trying the baby food?