Constipation..

So I know pregnancy causes constipation, but every time I try to poop it gets stuck! After I poop a little bit it just keeps getting a stuck feeling.. Is it going to hurt the baby if I push? I'm only 9 weeks and this is my first so I'm really nervous!