Anal play

So, last night, I was giving my SO a blow job. Well, I got really into it and started licking his butt hole. He kinda tensed up and first. And his whole reaction changed. Not in a good way. Well I kept going and eventually he gave in and relaxed a little. I couldn't believe I did it, but he held my head down at one point. Do all men secretly like this? I don't know how to bring it up because he hasn't mentioned it.