Sometimes I am extremely stupid

So my friend with benefits yesterday while we were doing it at one point he was like baby come please come but some odd reason it was like nope my body was enjoying it but not for me to come. So he pulled out and started fingering me but nope nothing happen. I feel my mind was focusing in other stuff that it was impossible. My biggest problem is the I know I am not the only he fucks so I think lately I been getting turn off. Is sad because we both enjoy each other and he is been my only one(he took my virginity) but I hate that he can’t be just mine.