my husband feels like his son hates him

Hey I'm new at this! My husband and I just have started getting visitation with his two year old son... so just a little back ground... we didn't know my husband didn't have a son till he was a month old... well his mom let us see him up until he was about 5 months old... so in the 2 year time frame we haven't seen him and she has "babied" him as in there's no disinclined at all. And he has never been away from her till now like not even a night till about two weeks ago.... so tonight we Picked him up from our sitters house and we are on our way home and my husband tells me that he's frustrated because he feels like his son hates him... because of the way his son acts when my husband walks in the room... how do I comfort him in this situation!?