On top?

Backstory: So I'm not a virgin but I also wouldn't refer to myself as "experienced" in bed. When I lost my virginity I was so drunk I hardly remember (I'm not bitter or sad or affected by this, so no need to worry or anything) but I know that I was on top at some point during that. But that's the only time I've ever been on top. I never really had an issue or cared much about that until just recently when I saw that I guy I slept with told his friends we slept together (again, not bitter, sad, or affected) but he also told them that I was "passive" in bed and this did sting a little. I didn't realize that that may be an issue or a turn off or anything until that point.