Losing weight during pregnancy

Hollie • 26. 2 1/2 yo daughter. Nurse. Live. Laugh. Pray & do yoga :)
I'm overweight.. Not like completely morbidly obese, but I need to lose some weight and I'm scared with this pregnancy I'm going to gain a lot like my first (50lbs) but that was because I ate like crap! I'm doing better this time around, only caving to bad food once in a while. I want to lose weight instead of gain which I think is okay as long as your not depriving your baby of nutrients! I'm doing everything I'm suppose to.. I'm starting to work out at the gym next week. Not vigorously.. Just walking and some light stuff to stay active. I'm almost ten weeks. Is this crazy or other moms kinda in the same position?