Possible Milk Allergy?

I have a 3 month old that is exclusively breast fed. He’s had some tummy issues lately. He’s spiked a fever and been to the doctor. He was absolutely fine yesterday. Today my baby sitter is saying that he is extremely fussy. Has anyone else’s baby had symptoms like this and it been a milk allergy? I’m super stressed out. He’s typically such a happy baby but he’s been incredibly fussy. I haven’t changed my diet or eaten anything out of the ordinary.

**pic for attention.