Help I am freaking out!!!!

I have such mixed emotions. I had an ectopic in June and had surgery. I had a persistent ectopic and needed a methotrexate shot in August. I was told I can start trying after three months. We weren’t overly careful near the 2.75 month period and I got pregnant. I had an ultrasound and it’s in my uterus but I am having constant panic attacks about it being so soon after my shot. I had three periods since my injection. My doctor isn’t overly concerned but I can’t seem to shake the anxiety. Help me please!!!! I am 5 weeks and 3 days. I am so scared right now.