Lack of orgasms...

I love my boyfriend a lot and sex feels good but it's hard not to have a little resentment when I rarely climax . There had been a lot of times where we are both really into it and he cums fast but I still want to keep going . It makes it really hard not to get frustrated because I try to get him hard again but he can't or isn't into it . He goes down on me ( I can't seem to cum even with that ) every time we have sex so it's not lack of effort so I feel bad for getting upset when I get so close to cumming and he has to pull out . For example when he woke me up to have sex in the middle of the night a few weeks back it was amazing and I came so close then he came and it was over ... I was almost in tears because I was so frustrated so I just went back to sleep . He knows I don't cum very often and he feels bad so I don't like to talk about it too much with him . I just thought by 1 1/2 years we'd figure it out by now although we don't have sex that often because we don't live together yet but still . Do y'all have any suggestions to figure this one out ?