Co parenting!!!!!

Okay so currently I’m stuck in a hard situation I need your guys help! My ex husband and me are co parenting ! Our kids are 4 year little girl and a 8 old boy! So last night my ex messages me and asks if it was okay if my kids go over a whole to spend at his house ! Keep in mind he works so I asked who would watch them while he work he said my girlfriends grandma! I wasn’t okay with it because I don’t know who they are regardless if they aren’t bad people I don’t trust them with my kids! So he got really upset and told me that I should have let them go ! That he had plans for them ! Do you guys think I’m being to self fish or over protective with my kids ? Like I understand he has needs time with them and it’s fine he does get them Saturday through Tuesday the days he’s off but I don’t just trust anyone with my kids specially someone I never ever meet in my life ! Idk what to do guys help me !