Am I Wrong or Is My Husband

So my husband wants my son to go to his ex wife's house for a sleepover with his sons. They invited him for his birthday. I don't want my son to go because he has issues with wetting the bed. He's 12 and I don't want him to get bullied. My husband than goes "You think my kids are bullies". I told him no I'm just worried for him. He says I'm sheltering my son too much. That his kids spend the night here all the time and chances are they already know, but they still invited him. He decides to go ask my son how he would feel about going even though I said I'm not comfortable. He said he wants to hear his opinion. My son asks "Will they make fun of me?". My husband goes "No they know I'll beat their ass for bullying". So now my son wants to go and I'm pissed my husband even asked him. His kids aren't bullies . I just feel something will happen. He keeps telling me I need to let him have fun and not let some silly little problem stop him. I'm still aggravated though.