how long does ice cream/ caffeine stay in your system?

I forgot I can't nurse after eating a huge ice cream...oops. Does anyone know how long I should wait before it's out of my system and I can nurse again?