Thinking of giving up on BF

I’m a FTM and the only person to attempt BF in my immediate family. My son is 2.5 weeks old. Since he was born he’s been eating more than what is recommended - they told us a few drops of colostrum would fill him and that never was enough. We struggled to keep him full until my milk came in. I’m now producing about 4oz on each side if I pump. The last few days he’s been eating nearly every hour. The dr wants us to wake him up to feed if it’s been 4 hours since the last feeding. This means at night we are constantly up. I just don’t know how much longer I can do this for. I feel like I’m constantly feeding him or pumping. He’s gaining weight and is totally healthy but he’s a big boy and I just can’t keep up. I don’t know what to do now because I’m considering stopping BF and just giving him formula.