Allergic to Condoms? Maybe.

Tristan
I'm not sure what it is in them I'm allergic to. I can wear latex gloves whenever I'm dying my hair or something and there's not irritation. But if I have sex with a condom it burns and gets really raw and I have to just sit in the bath for an hour afterwards. 
So me and my partner don't use protection as of now, which is bad I know, but I'm about to start the pill when I have enough money. 
Has anyone had this problem and are there any other solutions besides birth control?