Weight gain.

Has anyone else had trouble gaining weight? This is my second baby and I am 20 weeks and have gained 4 pounds total. With my first I gained a total of 20 pounds but I gained mostly in my 2nd and 3rd trimester. The 4 pounds I've gained this time have all been before 13 weeks, since I havnt gained anything. I do workout but nothing serious and I eat plenty.. I just don't understand why I am not gaining weight. I have an OB appointment June 4th, just wondering if any other moms out there have the same thing happening?** No rude comments please!**