Should I try for a baby?

Me and my husband haven’t been marry very long about half a year. I found I was fishy with PCOS which makes my acne, weight loss and gain, my facial and up and my irregular periods understandable. I was told I may have a tough time conceiving. Me and my husband have decided to try for a baby, since we will already have a tough time we don’t wanna push it out and make it even harder. Since my husband hasn’t been into showing me love. Yes we have sex but he doesn’t notice me anymore. He has his own issues but now the narrative from others says that I should try because he is not ready. But I am 100% like I’m doing it for me not him but is that selfish should I even try anymore or should I just stop. I feel so lonely all the time but a baby is what I want more then anything and I know I can handle but I don’t know if he could. What should I do? Should I try?