Condoms dry you out?

Sevina
Whenever me and my boyfriend have sex with a condom I always seem to feel dry down there after a while and my labia lips start to hurt if he goes too fast. When we have sex without a condom we never seem to have that problem because I stay wet. Does this happen to anyone else?
P.S. I have an IUD so we don't need to always use a condom but we usually use one so that he can last longer