Baby acts like he hates dad

Di
My baby is almost 7 weeks now and in the first couple of weeks he'd be fine with his dad and even fall asleep on him. Now my boyfriend can't even talk to the baby without him screaming. He's fine when literally anyone else holds him or when he lies alone sometimes but if my boyfriend picks him up he acts like my boyfriend is trying to kill him. The baby is breastfed so that's a factor but I don't understand why he's fine with people he barely sees but not someone he sees every single day. It's starting to get to me because I can't even use the bathroom in peace because I hear him crying in the background. Does anyone have any tips to get him relax a little when his dad has him?