So frustrated !!

My son just turned 3 months old and has been EBF this whole time. A few weeks ago he started getting really fussy when he'd nurse. He'd latch for 1-2 min, pull off and scream and refuse to nurse. After a few days of this I go worried he wasn't getting full so I pumped a bottle for him and he guzzled it down like he was starving. 3 hours later, he the as hungry so I tried nursing him and he wouldn't latch and was screaming, offered a bottle and he was good. After a few days of this I decided to exclusively pump and bottle feed. Well, now my son is having issues with bottles. He nurses at night no issue, then during the day it's a fight to get him to eat both from the bottle and the breast. He takes 1 oz out of a bottle and will scream and refuse any more and then 3-4 hours later the same thing. When I offer the breast he nurses less than 3 min and starts screaming. I've checked his mouth and don't see any problems, my breasts don't hurt so I don't see I have an infection, he's having at least 5 wet diapers a day and looks fine but feeding time is becoming a nightmare and he's just not eating much at all. Help !?