Husband hiding masturbation!?!

So in a random convo I asked my husband (married only a few months) when he masturbated last (he said he hardly ever did it and I know because we spend our time together when not working and through the years we have dated he's only done it once in front of me). He said he didn't want to answer because it's too private! WTF! I'm your wife hello! He finally said three days ago so I guess he does this (often) in the shower! Besides us TTC and doing this so often I'm just so upset that he hides it from me. I always thought my marriage would be based on communication and trust- plus he's not that affectionate so I'm just annoyed with everything. I'm not sure if it's the holiday stress that's compounding this or my hormones but I'm really feeling alone in my own marriage at the moment. 
Any marriage advice from couples who have been married a long time? Thank you