I feel guilty.!

Michelle • 27 years old Wifey & Momma of a 3 year old. Currently expecting #2 in July 2015
I started out in the pregnancy drinking no more coffee and soda. Then once I got into my second trimester I start occasionally having a cup of coffee or a small Starbucks latte with half the caffeine. Now all I can think about is coffee. I want it daily. I don't but I do have coffee a couple times a week and I feel bad. Is this gonna hurt my baby if I drink coffee the remainder of my pregnancy?