Can newborns tell who the mother is?

Whenever my one month old is fussy, I noticed whenever his dad or grandma holds him he doesn’t stop crying. As soon as I hold him (the mommy) he automatically stops. I’d joke and tell the hubby it’s because he knows I’m mommy but actually curious, does he know?! I would assume he’s too small to understand.