Time between pumping/feeding

What's the longest you go during the day without nursing or pumping? I nursed him at 8:30, he had a bottle while I was gone, and then I planned to nurse him when I got back. It's 12:00 now and he's asleep. I didn't bring my pump and I don't want to wake him. Will their affect my supply? Am I being crazy? 😕