The pill & Sex

I've been on the pill for a week now and the next time me and my boyfriend have sex I want it to be without a condom. But he's not into it because I had a pregnancy scare earlier this month. Is it too soon?