FTM and rant !!! kind of mad

Ok, I am a sham with a 6 month old who does really like to be held by anyone but me and maybe my husband so time. My in-laws call him a brat and mama's boy ( like it's a bad thing if my baby to prefer me). He cries if any of them hold him or sometimes talk to him. My husband told me that I don't allow anyone else to comfort him and it's all my fault that he don't like other people. I wait unless aleast few minutes before going to get him for them . He nevers calm down until I pick him up. I refuse to let my baby cry. If he don't want you to hold then I am not going to make him stay with u. I have try sitting next to them to see if he warm up but he never does. We go by their house once a week for 5 to 7 hours at time . I feel like my blaming me and told me to fix it. What do he want me to do?