Does anyone ever feel like they have ripped their partners off?

just a little more into the question, sometimes with pcos I have my down days where anything and everything seems to bother me. lately it's been that I feel like I have ripped my husband off from having kids. He doesn't see it that way nor does he care he always says, but I feel guilty because we are struggling and have been for 4 years. All we ever wanted was a family of our own, but that just seems impossible. I feel like I gave him the sh*t end of the stick and all I want is to have this mans children because he is the love of my life. I just want to know if others have felt this way?