Are we being selfish?

A little back story, me and my so have ttc for 6 years, we were recently called selfish for trying to have a baby when both me and my so have health problems. I have ME, CFS, IBS, pcos, abdominal & head migraine since being 3, depression, and a mental breakdown along with some other problems thrown in, my so is chronic asthmatic & is being tested for brittle bone disease. Theres also a long line of heart disease, heart attacks, strokes, brain tumours, kidney failure, and a few others on my side of the family all been passed down, on his side there is similar and every female on his mams side (apart from her) has battled & died of cancer that was also passed down, I've recently been diagnosed with a relatively new health condition that was in my mams genes and we have broken all research on it, I've been told I will pass it onto my child when they hit their 20s. My so & I want a child more than anything but are we being selfish to want this knowing they could have a lot of these health conditions?