Workouts during first trimester after IVF

Br
My <a href="https://glowing.com/glow-fertility-program">IVF</a> doc has instructed me not to workout (light walking only), and it's killing me! I'm 6wks 5days & am worried I'm going to gain more than necessary at this rate. What are you ladies doing? Have your docs also advised you to do the same? Can I start to workout in the 2nd trimester at least???