HSV-1 and sex!!

Hey everyone! So I couldn’t find ANY information on the internet about whether or not you can responsibly have penetrative sex (M+F) while infected with a cold sore? I have had HSV-1 for years and never spread it by being super safe, I understand that anything mouth related is OFF THE TABLE. but... theoretically could sex still be safe? (given there is no kissing or oral?)