Is heavy manual labor bad when you’re TTC?

I work on a ranch and am expected to lift things well over 50lbs, drive equipment, dig post holes, etc daily. I am SO nervous that it will affect me getting pregnant or whenever I do get pregnant that it will hurt the baby. Does anyone have any insight on this or am I just being paranoid??