PEANUT ALLERGY

My family wants to eat at Texas Roadhouse (which I do love) but that is known for having peanuts all over the place,especially in the waiting area. Although peanut allergies don't affect anyone in our family directly, I have 3 cousins who are truly deathly allergic to them. Their parents also do not have an allergy. Should I take my 2 month old son into this restaurant anyway? Am I just panicking for no reason other than I'm a parent? (I have a 5 year old also)