Step kid

So my husband has a step son. When he got with his ex she already had a kid of 3yrs. So be basically raise the kid. I have no problem with his son coming over but his son doesn't want to come if his big brother doesn't come with him. I already talked to my husband about the older kid coming over (he is now 9 and his son is about to be 6). He said he loves the other like his son. But I don't want him over knowing he is not his son.

Do you guys think it was wrong for me telling him I don't want the other kid over.?