Breastmilk question? (Just a theory)

Please don't call me stupid because I literally have no idea if this would work and I'm a little fearful of breastfeeding while visiting at others houses but I don't want to use formula! Do you think this would be okay? 
-put storage bag of breastmilk in a container or small lunchbox with an ice pack to keep it cool (refrigerated I guess) 
-put container/lunchbox in diaper bag
 
-take it out later when baby is hungry and I'm at a friends house/ families.
 
-warm it up there in a bottle for baby to feed.  
I see no flaws in this system instead of needing to take time alone to breastfeed while I'm trying to be with family and the baby. Any problems with this as a feeding method? I'm sorry if this seems obvious as to if this is wrong or right! I'm a first time mom!