Baby sensitive to breast milk

My son started out on formula in the hospital. Once we were home I was pumping and also supplementing 2-3 bottles with formula for the first 2 weeks. He then started getting extremely gassy and colicky and was switched to alimentum. At that time I stopped giving breast milk so we could see if the alimentum helped his issues. I just tried to reintroduce breast milk at 6 weeks and by day 2 he was back to being extremely fussy, crying a few hours, and really gassy. Has anyone had something similar happen? Was your baby able to adjust back to the breast milk? I have a supply frozen as I have been pumping 2-3 times a day since he switched, I would hate for it all to go to waste.