his ex wore my favorite perfume?

When I first wore it around him, he wouldn't stop smelling me, and he told me I smelled like flowers. The next time I saw him, we went out to lunch, and he told me his ex wore the same perfume. 
It's my favorite perfume...but now I don't want to wear it around him :(