2nd trimester exercise

I am 20 weeks and have been working out nearly an hour 5 to 6 times a week, just continuing as I did before pregnancy modifying as I need to, yesterday at the gym I did a modified version of the Superman exercise of a exercise ball, without thinking about it. Then I got home and panicked realizing I’m probably to far along to be laying on my stomach to do that exercise. Could I have harmed my baby by doing this? I have been feeling so bad about it!