Condom burns during / after sex

I feel silly for posting this, but I can't find any answers anywhere. 
Whenever myself and my partner have sex, we always use a condom, but we both have a really bad burning / redness 'downstairs' after using one. 
Would this mean we are allergic to the lube on the condom or the comdom itself?