Weight gain/loss throughout pregnancy

Erin
Okay so I'm just curious, I'm about 15 weeks pregnant and had lost about 15 lbs in the first trimester.  I was told don't worry about it.  Now that I'm in the second, I gained some weight back once I got my appetite back, however I've now lost those 5/6 lbs I had gained back.  Family and friends say not to worry it's normal but I just want other opinions.  Should I call the midwife or just keep eating healthy and not worry?
Thanks!