3 Month old Stranger Axiety?

Je

My LO just turned 3 months today (yay!). But for the last few weeks (started around 2 months) he has only been letting my husband and I hold him, sometimes he won't even go to my husband. My mother lives in my house and he sees her every day and he still won't go to her. He screams like he is in pain. He will occasionally go to my sister and allow her to hold him for maybe 10 minutes and then he screams until I take him back. Every weekend we go out to family's homes or they came to ours. He will become inconsolable at these family functions even when they don't hold him and are just talking to him. My husband and I do have shyer dispositions but my LO has seen these people often since birth as we are all very close and I always thought stranger axiety starts at 6 months. Is he just over stimulated? He does have a milk protein allergy so I always thought he just wanted me because he was in pain but since we have changed his formula he has been a very happy baby at home so I am at a loss. Any suggestions or advice would be greatly appreciated.