Lifting in early pregnancy...dangerous to baby or no?

I am 10 weeks and 1 day today. I have been avoiding lifting anything over 15/20 pounds just out of caution, but today one of my kids at work fell and hurt himself and just out of habit I immediately picked him up. He weights probably 50 pounds. I put him down as soon as I realized what I was doing but I have completely freaked myself out now!

I have read different things saying lifting is dangerous, other things that say it’s only dangerous because mom might pull something. So I’m just curious!!