is a condom necessary?

i’m on the pill and take it religiously. we still use condoms but I want to ditch the condom and use the withdrawal method. is that okay and am I still being safe?