He doesn't listen

Ro
I usually never openly complain about my partner but this really hurts. 
I hang onto literally every little detail about what he says and what he does. 
I could be in the car with him or just talking about something I love or something that I'm passionate about and I can literally see the disinterest in his face. He nods his head and goes "mmm" and it's so obvious he's not listening or doesn't care or thinks what I'm saying is what every girl says. Even one time I called him out on it and he admitted to thinking of something else.... like occasionally? That's fine but like it's constant... 
and he doesn't hang on to details that matters... he doesn't listen when I talk... he doesn't know how to touch me affectionately... doesn't know what to say...  but he remembers such specifics when it comes to sex... like he's so interested with sex and could talk about it and listen to me talk about it..
I feel like he just wants someone and he sees I care. We're very happy when loving on each other and being a couple but... I'm not just some person... 
what do I do...