Hmm i need help..

So recently me and my boyfriend were talking about our past when the subject turned to 'sex'... I was nervous about telling him because I thought he might look at me different.. But I soon gained the courage to tell when we soon switched it up to the question

"What if we were to have sex"

He told me he was a virgin but he doesn't value sexual intimacy. It was great knowing that its not what he wants... But, I soon thought... Is it different for someone to take a persons virginity?

What should I do?