My now 3 week old is very easy to handle.....when my husband is at work.

We wake up, change diaper, eat, and back to sleep... but as soon as my husband gets home baby is wide awake and crying hysterically for no apparent reason and settles down at bed time. Is this normal for a baby? Due to work my husband gets no real time home with us and spent his 10 days of paternity in bed because he caught a virus that had been going around.... is it just lack of bonding? It's really starting to upset my husband because he thinks his child hates him.