Is 3 weeks to early to leave my baby overnight?

It's my birthday this weekend and we are having a small gathering at my house and then going out to the town afterwards with a group of our friends. My partner offered to stay home and look after our 3 week old son, but my dad and step mother offered to have him on Saturday and night and Sunday until we are ready to pick him up! It's so generous of them, and I definitely trust them, but I feel a bit guilty for leaving him with non parents for the whole night. Should I accept their offer, or should I tell my partner to stay home? I would love to have him come with us to make it the perfect birthday I will just feel sad for my son...