Should I take it easy from the gym when ttc?

I do intense plyometrics, weight training, and cardio daily. My husband and I have been ttc, I don't ovulate, I've had tests done to figure out why but no known cause. Do you think the exercise has an effect on my body?

I'm going to take letrozole to force my body to ovulate, just not sure if taking it easy from the gym would make a difference. (I'm a gym addict so that would be very tough for me) any advice?