Is this wrong?

I don’t trust my father-in-law or brother-in-law with my child. The brother is very immature and doesn’t do anything his age yet and the father is just mean and doesn’t pay attention enough for me to feel comfortable. He offered to watch my little one for me so my husband and I could go out on a date for the first time in a while and my husband turned it down without talking to me first. Which I am sort of glad he did turn him down but at the same time, I got mad at him for not talking to me first and so now we are in a fight.. but it’s more than just that. We haven’t had a date or really any time of just me and my husband for a LONG time. It seems like he doesn’t even care anymore and when I confronted him about it last night, he flipped out on me and told me that we were going to finally have a night out just the two of us after we finished our obligations but that is out the window now because he was going to have his dad and siblings watch our little one. I apologized for being mean about his family and that I didn’t trust them, but he took it as me saying sorry to go have our night out. As he rolled over in a huff, he said whatever, I don’t care. As he was saying care, I said that the apology wasn’t to get the date night, but about your family and that I am just not comfortable with it. Every time we do get the chance to get out, with our little one, we never get to go alone. He invited his sister last time and the time before that, his brother tagged along and he didn’t even try to stop that.. he just lets it happen. He says he feels bad about these things but he never acts upon them and lately I feel like there is someone else, again.

He used to message females that showed a little interest in him when we were dating and engaged.

I don’t know what to do.. I slept on the couch last night, just to get away.