is it safe to use the pullout method if you aren't fertile?

During my period and during my fertile window I use condoms as well but otherwise we use the pullout method or go for a little while then use a condom before he comes