2nd baby fears

Calina • Married to a wonderful man. Proud mommy of Kylen 9/29/2011 and my Angel baby Adrienne born sleeping on 9/24/2009. Pregnant! due 9/7/2015.
My husband and I have started trying for another baby. We have a 3 year old son now. I know I want another baby, but I'm scared that I will be letting my son down, or that hee won't feel as loved as he is now. Has anybody gone through this feeling? How did you handle it?