Condom Rash??

Ca
My boyfriend and I started having sex in August 2016 (we lost our virginities to eachother) and of course at first it hurt and was very uncomfortable (he couldn't put it in without a lot of pain) . Starting a month ago it got a lot better and now I can take it np 😉 but the first time we had sex without pain it was rough and for the week after I was in a shit ton of pain and I had swelling all around my vagina (lips and around the hole). I put inflammatory cream on it and it went down after a while. But for the next three times after that the same thing has happened but at a less extremety. Here are the details:
Each time used :
a magnum xl condom
Lube that I've used before for tampons etc
I thought it might have been a condom rash because we had sex without a condom twice and it didn't happen. But we used a condom last night and it didn't happen. I know it's not a yeast infection 
Ha this is the first time I'm posting