Allergic to semen?

So for as long as ive had sex if i get any come in me of sometimes just pre-come, after like half an hour at most my vagina will get swollen and painful. It doesnt happen if i take benadryl. I got an allergy test done and asked if there was a way to see if i was actually allergic to it or if it was just a weird thing. They said to scratch my wrist or forearm and then put semen on it. So i tried that and there was no reaction at all that time but i had taken benadryl the night before. I guess what im asking is if anyone else has this problem, or if anyone knows anything i can do about it. Besides the obvious which would be no sex or condoms always.