Should we break up?

I am 37 and have been with my boyfriend (he is 40) for 3 years, but have known him for 15 years. We started ttc #1 for both of us in September 2016. He was excited and told people we were trying. I went and got tested in December 2016 and had uterine polyps removed, but other than that hsg normal, tubes open, blood tests came back normal " for my age" Well, we have tried all of 2017 and when I approached him about getting tested he said he will not. He said he will get healthier and we will see what happens. In the meantime, I froze more eggs in January because I feel time slipping away and really want children and always have and he has known this since day 1. Now that it may not be 'easy' it seems like he doesn't care if he has kids or not. Every time I hear of another pregnancy, it gets harder to feel like I am being left behind and for him to not want answers or get tested is very frustrating. I feel like I do not have any more time to waste. Should we go our separate ways? I just don't understand why he wouldn't want to know why 85% of couples get preganant within a year and we didn't?!