How to politely ask neighbors to stop coming around at wrong times

Today my husband and I were doing yard work and we have all 3 kids out playing on their own. I had not cooked and it was 5:30 when our neighbor and his 4 year old kid stopped by. As soon as he got there I told him it’s been a busy day and still have things to do include cooking. He replied with “oh really”

His kid just goes into our garage and starts taking toys out and not even 5 min in he is asking to go potty. (They live 4 houses down) I understand kids go when they have to go..I have a 4 and a 2 year old that our potty trained. I asked the dad do you want me to Take him since EVERYTIME they come kid needs to use the bathroom and they come in our house with shoes on and pee all over the toilet even though we have asked them to take off their shoes. He said no it was fine.So I said well I’m going in to cook and took my baby.my husband stayed outside with them to grill the steaks and I went in to do sides and prepare the steaks.Kids kept on opening the front door and leaving it open so I closed it and locked it. Back door was unlock and hubby has his keys. 10 min later I hear knocking. I could not open it because I was in the middle of changing a poopy diaper. Knocking does not stop until I was done changing my baby. It was the neighbor trying to bring in his kid to the bathroom…

How the hell do I politely ask not to come during dinner times and to ask their child if he needs to go potty before they leave THEIR house?!! I feel like they are always imposing and making themselves at home. I’m sick of cleaning after them. I’m sick of having to drop everything and entertain them. I would hate for our kids to lose one of their little friends.