Month 3 of TTC & thinking of giving up

I decided I didn't want kids until recently (another baby boom in my small town gave me the case of baby fever), but I'm just annoyed at how easy it seemed to get pregnant when I wasn't trying. 

I got pregnant with my son within 3 months of being sexually active with the guy.

Then by the time my fiancé and I got together it didn't take long for me to get pregnant. I've been pregnant 3-4 times within 2 years but I chose not to have kids (for my personal reasons- my choice I shouldn't have to explain myself). But now that things have changed in life and I'm a stay at home mom for the first time, I'm actually into the idea of having another child. But all at the same time, now that I spend every day charting & recording my info with multiple charts it seems like none of my baby dance days & ovulation days seem to coincide with eachother and I'm started to think I should give up. Maybe it's not my time to be a mom again. 

Anyone feel like giving up only after a few months of trying? Or is it just me? All I know is my fiancé wants a kid of his own (but will still love my son no matter what).