Pregnancy paranoia

Before I found out I had Baileys liquor in the mornings and coffee and had a glass of beer and wine on 2 occasions. I haven't had caffeine or alcohol since I found out a week ago.  I'm so scared now that it will affect my baby and will damage him in some way. And I won't even find that out until much much later!