All he wants to do is nurse

Pretty much whenever I hold my 3 week old all he wants to do is nurse. If someone else holds him he will fall asleep in their arms. Not with me though, he only wants to eat. Is this normal?