He never wants me

I love him, but I’m frustrated that HE is never in the mood. Weeks can pass by and he doesn’t touch me. We have only lived together 3 years and have a toddler. We are in our early 40s but to me this is not normal. I’ve talked to friends and they have sex regularly like 3 times a week. With my ex husband of almost 20 years he wanted me all the time. I don’t know what to think sometimes. I’ve even have thought maybe he’s not attracted to women because I’m not unattractive. I just don’t get it. Talking to him doesn’t help because all it does is leads to arguments. We went to the doctor once and his testosterone was normal. Also, he goes to the gym regularly and eats healthy. I just don’t know what to do. And yes I get sex is not the most important thing in a relationship but it’s still important.