Husband... ?

The other night I had tried to get my husband to come to bed and have sex. He said that he was tired and just wanted to watch a show on tv, so I went on to bed without him. He finally came to bed but I didn't sleep well that night and it was close to 5 am before I finally got to sleep. The next day he tells me that I must have been exhausted and that he tried to get me to wake up but I didn't. I kinda laughed and asked why he was trying to get me awake. he responded with "Well I wanted you to enjoy it too. I thought you were waking up, but then you didn't so I just finished and got up". I was shocked and didn't really reply. We did have consensual sex later that day and the next because we are TTC and it is my fertile window. I am just unsure of how to feel about the whole incident. I don't think he was doing it in a hurtful or controlling manner, but it still makes me feel kinda used and taken advantage of.