Sex & condoms

When me and my boyfriend have sex with condoms I always feel weird after , like I always feel like I have a yeast infection afterwards but when we don't use a condom I'm fine .. Is it possible for me to be allergic to the condom ?