Sleeping Positions

I’m only 7+3 weeks and I always fall asleep on my left side. Although through out the night I guess I move around, because I wake up on my back or right side. My first OB made a big deal about how I slept, so will it hurt the baby when I sleep on my back or right side?