Condom? No condom?

Me and my ex have been working on trying to fix things between us and we are working towards getting back together. I'm planning on flying out to see him and we had the decision about if we did sleep together are we using a condom or not? I told him I would feel more comfortable using a condom and he says it doesn't feel the same with a condom and he always pulls out. I'm on the pill and I've been on this pill for almost a year and I've been on BC for almost 4 years but I'm not sure if it's a good idea or if I'm just being paranoid. What do you guys think?