Dropping AM pump?

Mi
I nurse my baby (6m) around 530am, then pump after she's done, then leave for work at 630am. Before I returned to work, I would get a lot of milk (6-10oz). Since being back, I only get drops to .25oz. I assume my supply just regulated. When I pump during the day at work I get plenty for her & enough to freeze extra.
I'm considering dropping my morning pump so I can get to work 15 mins earlier so I take a longer lunch & get back to doing yoga over it (Bc who has time for themselves when they get home ;). 
My only hesitation is, if I drop this pump, would it affect the amount I am getting during the day? Any working moms drop their morning pump, and did it effect what you pumped during the day at work? I know, I'm being crazy paranoid! Lol