Torn about feeding solid

My son has been eyeing my food, chewing when he sees me eat, reaching out to grab and opening his mouth when a spoon is near for the past 3 weeks. I’ve been reading that it’s better to wait 6 months but I feel like I’m torturing him. I gave him a piece of banana to lick and he started crying when I took it away. He will be 5 months on Sunday and is EBF except Tuesday and Thursday he gets 12 oz of formula at daycare. he also has good neck and core strength but doesn’t sit yet by himself. Help a mama! Is it that bad really to feed him purées at 5 months?