Second thoughts of 3rd!?!? Please help!

I have always wanted 3 or 4 kids. My husband was happy with 2. Dd will be 5 in sept ds will be 2 in sept. I had finally convinced dog for a 3rd so removed iud in June and ttc. However now that I and not on bc, and recently started being home 5 days a week with both kids I am so overwhelmed and I know I want more But i am so worried about being pregnant and having a Nb bc my son is kinda wild and so much harder than my 1st. Anyone else been thru this and can give me advice or ease my mind!!!?? I'm so torn I want them 2 1/2 -3 years apart but not sure if I should wait a bit longer?