Birth control

So 6 years ago I accidentally got pregnant on birth control it ended in a miscarriage at 7 weeks. Fast forward to now, we've been trying to conceive for 4 years with no luck we have been on clomid 50mg, 100mg, 150mg, we have been on letrozole. My fiancé has had 5 sperm counts done, I've had blood work, Hsg and everything comes back perfect. So what's wrong with us or me? I had a visit with my obgyn last week and she put me back on birth control to see if it would help I mean it's been 4 years what could it hurt to try anyways. So I'm just wondering has anyone else done this and had it help them conceive? We just don't have the money for ivf or anything of the sort so praying this helps, we want to be parents so much. Any comments would be appreciated.