Breastfeeding Troubles :-(

I had my son 12/15. He was latching well up until yesterday. Now he just seems annoyed and frustrated when I offer him my breast. He latches for seconds and pushes me away and starts crying. The doc said I must supplement a bottle after each feeding, since he had lost a pound since birth (this was 2 days after birth).

I am still offering him my breast at each feeding and giving it 10 mins. Since he really isn't feeding from the breast well I pump while my husband bottle feeds him. I really want to breastfeed him. Help!