Sex without condom

Me and my boyfriend have always had sex with a condom, but on top of that I recently got on the birth control pill and have taken it religiously for 3 months now. I make sure I always take it at the same time everyday so it is most effective. However, I still make my boyfriend wear a condom so I don’t panic and he does without any issue. Now I’m starting to get tired of the condom and want to have sex without it.. but I’m too scared! Is it safe for me to have sex without a condom while on the pill if he pulls out? I need some reassurance that it’s safe!!