My son keeps kicking me in the belly

I swear it's daily and it makes me furious! He's only 3 and I feel bad getting so angry but if something happened to my baby I feel like it'd be hard to not want to blame him. I feel like that makes me a horrible mom😓