Is it time for a BREAK???!!

Sooo I'm 23 I was diagnosed with PCOS when I was 13, but didn't really care at the moment. I was engaged at 19 and married at 21. My Husband and I have been trying for about 2 years. My periods are extremely irregular and always have been I can go months with out one. I never used birth control my entire life only when we first got married I got it (nexplanon) in April and taken off in July after deciding we wanted to ttc. So long story short I've been doing provera and clomid for the past 6 months each month the clomid dose was increased. Now my Dr doesn't think it's time to continue and wants to refer me to <a href="https://glowing.com/glow-fertility-program">IVF</a>. After convincing him to try one more round (which I'm currently doing) I didn't have to take the provera to "induce" my period it just came which was exciting I already took my clomid doses and since I can't really track ovulation my s/o and I kind of wing it. I have bought the ovulation trackers and just to make sure I didn't miss a day I used a 3 month <a href="https://play.google.com/store/apps/details?id=com.glow.android">ovulation tracker</a> in just one month. I never ovulated. Also according to the monthly blood work I'm not ovulating but who knows. My husband is 31 and he thinks we should take a break. Trying has been fun but also emotionally stressful the past 2 years of trying and seeing family and friends become pregnant while your struggling to has taken a toll on me I hate to say that I am happy for them but a little bitter especially towards the ones I know know I'm trying. (Is that wrong?? 😳) but anyways my question being if when I go to get my blood work done tomorrow to see if I'm ovulating and it says I'm not is it time to not give up but just give it a break???