Cuddling a baby too much

Obviously I love cuddling with my 3 week old baby and at points during the day I'll pick him up and lay with him for an hour or two but everyone keeps telling me I'm doing it too much and he'll just be clingy and I'm spoiling him, it makes me upset that I'm basically being told to leave him alone sometimes and not hold him as much, surely there's no such thing as cuddling a baby too much is there?