Should I be Worried? Only a Yolk Sac, No Baby

Hi Ladies! So I miscarried last year and the year before that. Honestly my biggest fear is being pregnant and going through that again. I have been nothing but nervous since I found out I was pregnant and today I went to the doctor for my first ultrasound. We couldn't find a baby, just a yolk sac. Should I be worried? I would really like a heads up so I won't be as heartbroken this time. I'm 6 weeks and 1 days according to my lmp but the nurse said my sac measures exactly 6 weeks. She said come back in 2 weeks for another one... I'm just so nervous