What's your opinion?

So me and my boyfriend were messing around yesterday night and I was grabbing him down there and we were having fun and he moved my hand down a little lower and my finger was near his butthole and he squirmed and I moved my hand and he asked for me to do that again. So, I did. He liked it. I was a little freaked out at first, worrying why he liked that so much, and we talked about it and he was worried that I was thinking he was gay. 
Is this normal for straight men to like a little anal play? Have any of you had this experience? I'm just way curious.