I'm so lost.

Baxter • Preg with #2 after four months of active trying.
So I've been told before that I'm not affectionate enough. So I made sure to be.But now I cry at night several nights a week for feeling rejected and unwanted. Should I stop being affectionate again? I don't know what the happy medium is! I feel like I will shortly be bitter and cold due to my husband always rejecting me. He says "what's the big deal. How can you be   offended" 
​He knows and agreed to TTC but doesn't want to hear any details or info. Now he says "maybe I have low T" but when he was cheating the excuse was "maybe I have high T." I don't want to offend his manlihood but I'm sick of torturing myself.