Trying for a baby

Me and my husband have been together nearly 6 years and married for 2 years. We have a beautiful 2 and a half year old together as well as my husband has 2 children (12 and 7) from his previous relationship. When I had our son I had really bad post natal depression and was adamant I never wanted anymore kids. Since then I have got married and managed my mental health and got my career back on track. I was On and off for a while about having another baby but then in March this year I was diagnosed with stage 3 endometriosis which I was told I will struggle to conceive. They said in the near future I will need a hysterectomy. Since then I have wanted a baby so bad. I just want to complete my family and I don’t want my decision to be taken away from me for medical reasons. I want to make my own choices and try myself. My husband has also been on and off about having another baby. Our son is type one diabetic so his biggest fear is another baby being diabetic. Which I’ve said we will figure out how to cope if it ever came to it. I have had my contraceptive implant taken out due to it making me poorly. But my husband said the other day we can have another baby but then said to his sister today he says it to shut me up. I’ve tried talking to him and he jokes about it and says he will do whatever to shut me up or keep me happy. He just constantly says it’s all on me around people. If we do have another they are all gonna think I trapped him. Question is would you guys say it’s a bad idea me having another baby? I’m scared of

People Thinking I’m forcing him. He knows I’m not on contraception and we still have sex. He says he will have another baby. I don’t know what to do. I’m scared if we don’t have one now I’ll never get chance to have another baby 😭