Please read! Desperately need advice!

So I’ll try to make this as short as possible:

Currently I’m 30 years old. Partner is 32. He has no children. I have two who are 10 and 6. They were easily conceived in my early and mid twenties with no issues. I’ve always had irregular periods but they were 27-43 days tops. Still, was able to conceive them. 2016 rolls around. I go through a divorce and get with my current partner. We get pregnant immediately after I take out my iud. Unfortunately it wasn’t planned and my FIL died traumatically and I found him. Anyways long story....so I had an abortion at 6 weeks because of severe emotional issues and circumstances. We lived in separate states for two years and saw each other every few months. We never used protection during that time because whatever happened happened. Well nothing happened for a little over a year until I moved in with him. We got pregnant immediately and I had a MC at 5 or so weeks. Fast forward to now. 9 cycles later, still no pregnancy. Well I got diagnosed with PCOS (only suspecting because of irregular periods and mc and weight gain) bloodwork is pending and ultrasound looked good. Dr. Wants to put me on clomid but I really don’t want to do that. I’ve lost weight, confirmed ovulation last cycle, and opk’s are looking more normal now. Should I proceed with clomid or see how my weight loss affects my fertility? Please help!