He sniffs my underwear when I'm asleep

Should I be weirded out that I found a stash of my dirty panties in my fiance's nightstand? He said he likes to smell them when he masturbates either when I'm asleep or not home... Just not sure what to make of it!