What do you all think

So my husband will be traveling in September and insists he want to take our 3years old so with him to this 1month vacation to jordan. He has proven to not be responsible with the baby. He doesn't feed him or take care of his at all when I'm at work I come home at midnight n the boy is like starving, he's had issues changing his diaper n I kept battling diaper rash, he also would sleep all day n not attend to the baby, I mean nothing seriously. We have discussed all this before but this never improved. I managed to get him of diapers so he no longer suffers diaper rashes. I worked while he's home then when I get home he goes to work. I don't feel comfortable allowing him to take the baby. He's already said my parents are there to take care of him. This is the same women who I got into an argument with Cuz she refused to change my son's diaper n said it was nasty mind you I was very sick for a day with food poisoning n couldn't even move. I'm talking about my mother in law. Am I being to selfish for not letting go him take the baby on vacation? Is this wrong? Honestly speaking I don't trust him or his family with my child, I feel they have all given me valid reasons to feel uneasy about my baby traveling there. What do you all think I should do.