Should I be worried about supply

I took my baby to the dr this week and he's down to the 30th percentile on weight, he started at 90+. In the beginning I was pumping 5 oz a session. I then got the Nexplanon and it went down to 3oz, then 2, and now I'm luck to get half an ounce. I know I shouldn't base how much I'm making by what I'm pumping, but that mixed with his inability to gain weight at a rapid speed has me worried. I have about 70 oz stashed and am a stay at home mom, but he's only 4 months old so I'm worried that won't last long. I like to have a babysitter for my once a month girls night. 
I'm trying to intake my water consumption, eating oatmeal EVERY DAY, and pumping after baby goes to bed. I just feel a little lost. Any help?
Side note:  I'm using a nipple shield, would weaning him off the shield help??