So confused??

So y'all... my boyfriend has a very low sex drive so he will go like a month rejecting me. He has always been like that and I have gotten use to it. Well he uses the condoms to jack off, well he has used like most of them to jack off but can't have sex with me? Like that makes no sense and it makes me feel like he isn't attracted to me. Someone tell me I am being crazy or would it bother y'all too?