breast feeding failure.

My baby was born at 4 pounds 11 oz, and then he dropped 4 oz and came back up to 4 pounds 8 oz and stayed there for a week 😔 they are having me breast feed, and then give him a 2 oz bottle of my milk and then the next time just a bottle. they said it was because he wasn't latching right and he was using all of his energy to eat, but now I'm afraid to breast feed him for too long and I also don't have enough milk to keep up. I can't build a supply or put milk in the fridge because what I pump goes to his next feeding. How do I start to build a supply? And how do I produce more? Help would be appreciated, I feel like I failed him