Cold sore and herpes😫

I'm kinda embarrassed to post this but I'm worried now and cant stop thinking about it.

Ive had cold sores since I was young, like before I was even sexually active. Same with my SO. Whenever he gets a cold sore or when I get 1 we dont kiss or anything. If he has 1 I dont want to get a cold sore from him and vice versa.

But then last night my SO was wanting sex but I wasnt in the mood. So he tries to get me in the mood, and went down on me and started to eat me out. And like a minute later he stops and said he had a small cold sore in the corner of his lip.

I tell him to stop right away and I go and wash my area with soap and water.

I'm freaking out, because what if I get herpes from his cold sore?😫😫

Is there anything I can do to make sure I dont get it? Or is it to late already?

I get cold sores and they are annoying, and painful enough as it is. I definitely dont want to get herpes now!

Edit**

I guess I should have said I don't want to get genital herpes too.