Symptoms come and go

Hello, so I've miscarried twice and this is the first pregnancy I've had symptoms with. I am 5 weeks 5 days but some days (like today) I hardly have any symptoms, usually just nipple soreness hangs around. I've also had cramping off and on since before I even found out I was pregnant. I went to the er last week and it was fine for what they could see but should I still worry that I am going to lose the baby?