To be a stay at home mom or not?!?

JK
I think this is going to turn into a long post, so bear with me. Also please note, that the things I say are just meant to be an expression of my own feelings for my family not a judgement on anyone else's.
I've always planned to be a stay at home mom. I was raised by a sahm & loved it. We didn't have a lot of money for extras growing up,
But we had our mom to raise us and all of our needs were covered. 
My husband and I waited and planned financially so that we could live on just his income. Our budget will be tight, but we have good savings for emergencies and are both accustom to a modest lifestyle . 
We can afford for me to stay home and I really do want to be the one raising my children, not a stranger.
But (you knew that was coming right) I'm feeling anxious, how do I know I'll be good at it or that I won't miss working too much? So here a little of the other side of the story. Im a very driven person, I graduated college in 3 years then worked my way up in my career to a place where I am in a position at 26 most people aren't until there late 30's or 40's. I love my job & im good at it. It also keeps me active in my community which is a priority to me. But it can be very stressful and have long unpredictable hours. And no real way to cut back my position to work part time or from home. I don't think I could be the person my clients need me to be and the mom I want to be with out failing at one or both. If I step out of my position and wanted to come back later I'd be starting pretty much at square one... Not where I left off. I've thought about trying to step into a lower position in the company that could be designed for part time, but I don't know if the income would justify being away. 
To all the sahm & working moms & those who tried one or the other and hated it. Do you think a career driven person, who adores kids (& is good with them 😊) can be fulfilled staying home?