Switch to formula?

So my son was born 6lbs 7oz and dropped to 6lbs 3oz in the hospital. At his 2 week appt he was still only 6lbs 3oz. They had me supplement with formula every other feeding and the next two weeks he gained 5 oz each week. I cut back on the supplementing because he seemed satisfied after breastfeeding. At his 2 month appt he was only 8lbs 1oz. Also he spits up a lot and is on Zantac for reflux. Doctor doesn’t feel like he’s gaining enough and wants me to supplement more. I feel like I just don’t make enough or maybe my milk isn’t fatty enough or has enough calories.. wondering if I should just throw in the towel and give him formula only? Has anyone else gone through this?