HSV-1 and sex!?

Hey! Theoretically if I had a cold sore and had penetrative sex (ZERO ORAL/KISSING INVOLVED) would it be safe? I have never tried because I was too careful but my boyfriend keeps suggesting it. I refuse to spread the virus so wondering if anyone has experience with this?