Should I give up or keep trying???

My husband and I have been wanting a baby for a real long time. We have been currently trying but it’s been hard. My husband was suffering from renal disease since he was 19yrs old. It came to a point where he needed a Kidney transplant. Dec 1st of 2017 he got a kidney from a living donor. It’s been a 1yr and 2 months since surgery and it has been amazing!! He has become a completely different man.

But Being that his body has been fighting this disease for so long, conceiving a baby wasn’t his body’s priority. This disease alone does lower sperm count along with the medication taken. We’ve been to doctors and they say we have hope. So we agreed to try. We knew it was going to be a challenge but we were willing to take it head on.

So every month I track my period, ovulation time frame, continue to eat right, prenatal, etc. But nothing yet. I don’t know what else to do at this point. It’s very frustrating. He is more positive than I am and just tells me “just let it happen”. But I guess I’m being impatient. It just a sucks cause the baby fever is REAL!!! I mean im ready, we’re ready!!!

What are your thoughts? Give up or keep trying?