Sexually frustrated

We have a baby together and most of the time at the end of the day I'm tired but since off the pill have been horny almost everyday. My husband works a lot and projects at home that need to be done. He's lost weight from all the stress btw. I've told him a few times, been turned on so he said why haven't I said anything. Today, we were supposed to have sex - as we are also TTC. He was mad about something and said he's going to bed leaving me high and dry. Not only am I frustrated but hurt. I understand that he probably wasn't up for it and that he's working a lot to cut him some slack. But I feel a bit neglected that he's not making the effort to put some time for just 2 of us. For now we have sex once a month. Am I being selfish? Should I tell him this or am I just going to be a burden?