TTC and exercise!

I suffer from PCOS and already have a hard time conceiving. I have been taking metformin since February and exercising regularly (6x a week) since the beginning of May. My workouts consist of about 30 minutes of weight training and 15-20 minutes of cardio a day. I wouldn't say that I do anything too crazy and I don't lift too heavy of weights, but my workouts are pretty weight centered.

I know that healthy lifestyle habits like exercising and eating well are beneficial for TTC but can working out hinder conception around ovulation? I've taken ovulation tests for the past 4 months and I know I've ovulated but I still haven't gotten pregnant. It wasn't until last ovulation cycle that I started thinking I should take it easy during ovulation and TTC.

So I'm ovulating today (I've been taking tests for the last 5 days) and plan on having sex tonight and for the next few days as well. Does a rise in body temperature from exercise have an affect on sperm? I know it can't hurt to take a few days off, but it's honestly really hard to not get my physical activity in each day 😕 anyone have any insight on this topic?