Should I let his gf round my Newborn baby

He wants his gf round our newborn baby and u really don’t I’ve told him no and he insists I’m very uneasy on the idea do I let him take my newborn for weekends or wait till hes older I really don’t want another women round my baby he’s been with his gf on and off for 2 and half year and slept with me when they haven’t been together he doesn’t have custody I do even if u move on u plan on not letting my partner around my child