Maybe gay?

So me and my boyfriend have been together for 3yrs. We are very intimate with eachother some may consider us FREAKS. Anyways over the years we have been exploring more sexually. Over the last 2 year's me and my man have been kicking each other's anal. When I would kick hos I noticed he would shiver and moan loud and put his legs up higher. I figured it was just being freaky. Well couple months go by he starts spreading his cheeks and wanting me to finger it so I did. Started putting my finger all the way in and he loved it. So about a year later were using my toys on me laughing and playing and I say I'm about to put it in your ass and he said ok so I did. He wants to do anal on me but it just hurts too much and I find no joy. He has only had me put a dildo in his anal once but he loves for me to fondle it and lock it. I guess what I'm asking is has anyone else experienced this or feel it weird? But by no means does he act gay or even really like gay men. I've asked a couple of friends and some say that's not right and some say it's men have a extra something that gets them off down there and he might just be "FREAKY". IDK