Cuddling with baby while sleeping

I know you’re not supposed to sleep with your baby, but am I the only one that can’t imagine my baby in the other room, in his crib all alone with no blanket? Sounds silly but I’d much rather the family cuddles in bed all night where we’re all warm and together, than trying to force my baby into his own room and crib too early. Anyone else feel this way?