Discouraged to TTC.. Sex problems..

Im completely discouraged to TTC anymore.. Because i feel i have to try too hard to make my man have sex with me.. He always tired from work, always busy, we have sex like once a week at most.. Twice.. And its just draining me.. I feel like i cant keep on with this TTC like this.. I think this is supposed to be done together.. What can i do? Any advice?