Baby daddy's ex issues

Li
A brief backstory - my ex (the father of my two children) and I have been separated for close to six years now.  For the last five years he's been dating this woman.  I saw how she interacts with my kids (they would stay at their house every second weekend) and she was amazing.  They would come home clean, bathed and fed all because of her.  She would ask to see them during the time they were with us.  She always made sure they did fun things while they were at their fathers.  Her family love my kids like they're their own grandkids and have often stopped us in the shops to have a chat etc. So my relationship with his partner was great, I have a lot of respect and admiration for her and am thankful for all she's done.
Fast forward to the day their engagement party was meant to happen.  She finds out he (my ex) has been messaging and flirting with a girl and making plans to meet up.  I wasn't super surprised considering he did that to me multiple times.  They broke up, she moved out.  The kids cried when they told me what happened.  This was maybe two months ago.  Their father has since had no less than four girlfriends in that time, but that's a whole different story.
I messaged the (now ex) girlfriend and basically said I was sorry to hear what happened, and if she ever wanted to see the kids then don't hesitate to ask because she had become a part of their family.  She has taken them out for dinner a few times since then, and will be again this weekend.
The problem is, I know this can't continue forever.  I don't want to tel her she can't see them because she loves them and misses them.  But at what point does it become 'weird' for her to keep seeing them when their father is already dating other women? My husband thinks we need to put an end to it at some point.  Any advice?