When did you lose the baby weight?

I had my baby over two weeks ago and had appointment at ob today. My midwife asked me if I lost the weight yet and I was like nooooooo. Then she’s like maybe by your 6 week appointment you will. What???? I thought it takes longer to come off.