Is anyone trying for a baby but a bit worried at the same time?

I have 2 kids from a previous relationship and my partner has 1 but we’d like a baby of our own.

We aren’t trying yet but may start this month. A big part of me wants a baby together but another part of me is scared and worried incase I wish we’d stayed with 3 kids between us instead of having one together too.

Is it normal to be trying but always wondering if you’re doing the right thing?

Tia

Xx