help! what is going on??

Erin
My son is 12 weeks old and screams when I try to nurse him. We have a few feedings where he doesn't scream but he will latch and let go the whole time and be done after like 7 minutes. I am already supplementing each feeding with 2-3oz of pumped milk and one feeding with 4oz of formula. As soon as I give him the bottle he's fine. He wasn't gaining weight so dr told me to supplement each feeding. 
Is anyone else going through this? I'm getting so frustrated. I know he likes the bottle better bc it's easier but the fact that he fights nursing is really upsetting. I'm at the point where I want to just go to formula but feel so guilty! He's also not taking more than an hr nap a day and staying up until 2-3am :( (we are transitioning from bassinet to pack n play and he's not loving it) so I'm also just exhausted. Any advice is appreciated!