boyfriend wont go down on me

ive been with him for almost 3 years and when we first got together we both had never done anything with other people so the first time he went down on me it didnt feel good and he was really upset about it afterwards. he never wants to try anymore and i cant seem to be able to talk to him about it without him getting annoyed that "sex isnt the focus of our relationship and it doesnt matter" im starting to develope some resentment towards him because of my constant willingness to go down on him even though i dont really enjoy it. i love him v much but i have some unsettling feelings towards the topic