Is it possible?

I had unprotected sex once on my fertile week & i thought i was ready to start my family, turns out the next day i started freaking out bad about becoming pregnant & how i had a high chance. I have GAD & when i stress too much or have my anxiety through the roof im more likely to feel my panic attacks in my chest, lately I've been feeling that way, i even get scared to leave the house because of how bad my anxiety is. My question is can i affect my body with so much stress like not being able to get pregnant or if i do become pregnant because of so much stress can i cause a miscarriage? Although im scared i wouldn't wanna have a miscarriage either..