Too early for clomid??

Le
What would you do? I need some advice.
I stopped the bcp on Sept 1st. Been actively trying ever since...but I haven't ovulated once. I still get my period every 27-35 days, but no ovulation (confirmed by bloodwork).
My doctor seemed annoyed with me for asking for help before the 12 month mark, but still prescribed clomid. I'm on cd 1 right now, and I have 7 days to decide if I want to take it.
Is it too early to take clomid? Does anyone just spontaneously start ovulating regularly? I'm also afraid of the risk of birth defects and ovarian cancer (yes I read a lot of stuff online....)
Any opinions would really help me make this decision. We want a baby more than anything, naturally if possible, but are willing to do this if it's the best option. I'm 26 and my husband is 31.