I feel like such a jerk

I tend to over think everything and this is what has me feeling uneasy today..

We have a next-door neighbor 5yo that is a bit of a bully and he's really intense so I like to keep our distance from them because the whole family is like that. Very nice people but they are too overwhelming for me..

I went to have lunch with my kinder and pre k sons today.

The neighbor kid is in kinder. His mom had been going to have lunch with him but stopped going a few days ago.

He saw me with my boys and came to sit with us at the table where parents sit with their kids.

She has sent me pics and videos of my kids when she has gone to have lunch with her kid.

So I sent his mom a picture of him with my son and her reply was "aw did he ask if he could sit with you?" And I said " he saw me and ran to sit with us lol" and she responded "oh, sorry" and I said "it's ok!" Then she replied "hopefully he kinda behaved"

I feel like an asshole for answering the truth, should I have worded it differently?

I wasn't mean to the boy at all, even though he called my son stupid while they were having lunch. I told him to please not say that because it is not a nice word.

And he has a hard time keeping his hands to himself 😓

He sat with us the whole lunch and was being obnoxious as usual and I did my best at staying calm although my anxiety was really bad.

How should I handle this if it happens every time I go have lunch with my kids?

Am I overreacting?