Herpes

Soooo I had a talk with my Dr awhile ago  because I have cold sores and have been getting them since I was a small child I'm in my mid 20's now.  Herpes virus 1
Well my partner got herpes virus 1 the same strain that I have but on his genitals before we met, he told me upfront and was honest about it and I talked to my Dr about it and she said since it's the same virus being 1 that he wouldn't be able to give it to me through sex because my body has built up anti bodies 
But I read mixed things about it on the internet about people with cold sores need to be really careful not to spread it to their genitals themselves ect....so I'm a little worried...I get a cold sores 1-3 times a year, not very often, my partner doesn't speak of his outbreaks and if he has them I don't really seem to notice.
Anyone been told anything similar?