Dr refused treatment until I lose weight

Caroline • Ttc for 5 years now. We just lost our sweet baby at 18 weeks due to pprom. Heartbroken but have been given the ok to try again.
I have been TTC for 3+ years. I have PCOS and hypothyroidism. Weight loss is a BIG challenge for me. I started in January with changing my workout routine (I hired a personal trainer) and started tracking my calories. I was losing about a pound a week. I was pretty proud of that. Then I went to see a new specialist about my infertility. He lectured me on my weight and told me he would not treat me until I lost 20 pounds. I was devistated. 
I went to my PCP and got on Contrave (you can not TTC on this medicine!) in 5 weeks I lost 15 pounds! So I've decided to stay on the medication for 2 more months and try to lose another 20 pounds before I start treatment. 
Has anyone else had a Dr refuse treatment until you lose weight?