Opposed to Oral Sex

Three things before I even begin, because this post will turn into something it's not meant to be.
No, my partner is not gay.
No, I would never force him to do anything he did not want to do. Ever. I'm not that type of person.
We are both in our mid 20's.
I know I am not the only one whose partner doesn't like to give oral sex, but I am truly trying to figure out why some men don't like it. 
When I bring the subject up (because I am not opposed to giving him oral), his answer is it doesn't appeal to him. I respect that, but I would like more of a reason than that. We're both virgins, so I'm sure the fear of the unknown is a culprit, but he'll never admit it. 
Thanks for any and all helpful comments in advance 😊