My brothers feelings are hurt

Okay so I work in a daycare that I've basically grown up in, my mom works there too and sometimes my 17 year old brother will show up because the daycare is also a church and he goes on Wednesdays. It was the end of the day and I just had my bosses baby with me and I let my brother hold him and the baby took a look at his hair and screamed so I took him back and I said jokingly OMG you scared him it's your hair (his hair is blue and sticks two feet off his head I swear) and he got this look on his face like his feelings were hurt because he scared the baby. My boss even said "it's just your hair he doesn't know what's going on haha" 
And his feelings are legitimately hurt! Like come on just fix your hair and come back I promise the baby will love you 😂
I just feel horrible because I know my brother and his feelings were really hurt. :/