Weird thing when doing doggy style

So sometimes when I have sex doggy style it hurts so much under my ribs on the side that we have to stop. It's so weird. Like it hurts every time he pushes in. But it's not even my vagina that hurts?! He's a big guy but I mean come on. I thought it was just air first but it doesn't get better. sometimes it helps with another position, sometimes it doesn't, but it's only when he's pushing in. Any idea why and how we can prevent this? I'm always so ashamed when it happens.