Baby wont settle for Dad.

My LO is 3 & 1/2 months old and wont settle for anyone but me. My partner works mon-fri 7am til 5 and so baby is with me every day. My partner is amazing when he gets in from work though and cant do enough for our son, wanting to feed him, bath him etc. However when it comes time for me to go and have some time to myself with a shower or a bath our LO will scream until i get back. My partner is literaly trying everything to calm him, ive even given him tips that always work for me but nothing works. As soon as i pick him up from my partner he will stop crying. I know my partner is thinking our son hates him and gets quite upset over it. Im unsure of what to do????