Toddlers tantrums and pregnant

I know the baby is well protected in there but should I be worried about my toddler kicking and hitting? He doesn’t do it often but it makes me nervous every time. He’s also bad about tackling and jumping on me as well.