Sex on the pill

My boyfriend and I always use condoms in addition to me being on the pill. I have an alarm to take it at midnight, and I always take it in the range of 10 pm-3 am. Sometimes I want to have sex without the condom and then use the condom when he wants to finish. We are both tested and free of STDs. Realistically, how safe is this in regards to pregnancy?