Flu shot during 1st trimester?

During my first prenatal visit (10 weeks) my doctor told me that I should get the flu shot that day. I agreed it was fine. But, as the appointment went on we both forgot about the shot. I had a message from my doctor later that night telling me to stop in anytime to get my shot. Instead of driving to that specific clinic for the shot I made an appointment at a nearby clinic that is on my way home from work. When I get to that clinic the Nurse who was going to administer the shot told me they would not do it since I was still in my first trimester. Now I don't know how to feel....should I be disappointed that my doctor suggested I get it so early ad it is unsafe or am I putting my baby at risk by not getting it asap?! Anyone else get the flu shot in the first tri? What did your doctors suggest?