Mad at my doctor

So my husband and I have been trying to get pregnant for a year and 4 months so I went to the doctor for a checkup and just to talk to him. When I was there they weighed me and I had lost weight and when talking to my doctor he brought it up. He said I see you lost weight that's good but your still have to much fat to be healthy enough to get pregnant. YES he said FAT. I looked at him and not wanting to be rude I said okay well I'm losing weight but even when I was fit and skinny I couldnt get pregnant and he just said well that was then we are talking about now. 😡😡🤬🤬 so I said okay well thank you. I have no idea how I should have responded my husband was really mad when I told him. How would yall respond.