Is a 6 yo allowed in a public restroom of the opposite gender with their parent?

Let me clarify. I took my son to a birthday party at the local pool. After the party ended, we went to the women's restroom/locker room. I'm a women. There was an old woman sitting down finishing getting dressed.

I told my son to go into a stall, shut the door and take his suit off and I'll bring him his dry clothes. This is lady says to me, "let them change right here, that's what it's for. They don't need to go into a stall". My red flag alarms went up, because why did this old lady want my son to undress in front of her? I said, "no, he's going into a stall to change". She looks horrified and says, "that's a boy!?". (My son has shoulder length hair, but most definitely looks like a boy). This comment pissed me off, but I chalked it off to her being old and from another time and kept my mouth shut. I turned my attention back to my son.

Her friend comes out from one of the stalls and they start talking about my son. How they are disgusted he's in here and that this isn't right. I couldn't hold my tongue anymore. I said, "sorry ma'am, but my son is 6 years old. I'm not sending him into the men's restroom to change by himself". She said, "why not? It's safe here. We're on the island", looking to her friend, "didn't you say there's no crime here". (Yes our island is safe for the most part, but that doesn't keep pedophiles from never coming here)

I said, "ma'am, once again, I cannot trust that someone isn't going to touch him or hurt him, if he goes in the men's restroom. It is my job, as his mother to protect him. I'm not going to knowingly risk his safety".

I quickly got my son out of there. She walked out right before us and went to a lifeguard complaining that I had my 6 year old son in the women's restroom. He said if he's with his mom, he can be in there. She got pissed off and started leaving. I said to the lifeguard, "she's been harassing us in the bathroom". She started yelling something at me while she kept walking and one of the lifeguards called her by her name and said, "go, just go, right now. " She's obviously a regular.

I was fuming. I have never had an altercation with a stranger before, but when she's talking about one of my kids, I COULD NOT keep my mouth shut. If there hadn't been so many kids there, I probably would have said more.

So my question is, can a 6 old child go into the restroom with their parent if they are the opposite sex, even though the parent is in the correct sex bathroom?

I need to know if I just yelled at an old lady for no reason or was I in the right?