I feel horrible! Please help! 🙏

I'm a FTM and my little guy is 3 months old. At his 2 month wellness check up, the doctor wasn't happy with his weight gain. He was gaining, but slowly. She wanted me to supplement with a bottle of formula or breastmilk after every feeding. I gave him two bottles of formula while I was trying to pump enough, but that's all the formula he's had. I feel so bad about it, though! I've been trying to continue pumping, but it's hard keeping up with his hunger. I breastfeed him first, then give him a bottle. He always takes it and doesn't seem full when I try without the bottle. I've tried doing things to increase my supply, but he doesn't seem content. Just earlier I fed him and pumped afrerwards and as soon as I was done he was hungry again and of course, I was empty so he took what I had just pumped. If I supplement with formula, am I ruining the benefits of breastmilk?? I feel like such a failure because I had my heart set on breastfeeding!