Breast Milk Supply

I'm a first time mom and I wanted to exclusively feed my baby breast milk as long as possible. My little guy has trouble latching on, so we feed him breast milk from a bottle (we talked to a lactation consultant and the doctors before doing this). My supply has always been low, I could pump enough for each feeding during the day but we had to supplement with formula during the night. Now, I've noticed I can barely get two ounces out when I pump (if anything comes out at all, which is most of the time). I try pumping at least every hour to two hours, but I mostly get nothing to come out. I'm only 3 weeks postpartum, can you lose your milk this fast? I've talked to the doctors and the consultant and they've said to just keep trying. Any advice y'all?