Bringing refrigerated breastmilk to room temperature

My baby gets one bottle of expressed breastmilk a day so I can rest. I usually pump earlier in the day and refrigerate the breastmilk right away. We’ve been warming up the breastmilk by placing it in a cup of hot faucet water but sometimes this takes too long and baby is screaming. Is it okay to take out the breastmilk a few hours before we want to use it so it comes to room temperature on its own? I know you aren’t supposed to do this with frozen breastmilk but I’m unclear if it’s okay for refrigerated breastmilk.