Breast milk

My son is almost 4 months old and he breast feeds all day and night except for 3 feedings. I use those to pump/eat and have my husband feed him so they can spend time together. For the past 3 weeks my breast haven't felt full like they used to and my son's stomach seems to be growling shortly after a feeding and he doesn't seem to be satisfied. I've had to start supplementing 2-3 ounces of formula after each breast feeding session (ranges from 5-10 minutes) so he will relax and be full. Is this normal or is my body getting ready to stop producing?