someone with hypothyroidism have become pregnant easily ???

I have PCOS since I was 14 years old and I started to take the pill for that reason. Now I'm 31 and I'm TTC but also I have hypothyroidism and I'm kinda freaking out getting scare because I can't get pregnant soon. Once, many years ago, a doctor told me "if you want babies it is gonna be difficult". Someone had a similar situation and didn't have problems to get pregnant??