Losing weight during first trimester

Before becoming pregnant I weighed in at 145lb. About a month ago I weighed in at 141lb. Just today I weighed in at 138lb, not a lot of weight has gone down and doctor told me not to worry but should I still be concerned? And what may be the cause of this