Concerned new mommy with non vaccinated nieces and nephews

A SIL and BIL have four children all under age of 6 who are not vaccinated. I was told by other family members I should not be around then and neither should the baby when he/she comes for awhile. Is this true? I still need to do my research but curious what others think. I don't get along with this part of my family and I'm afraid they will think I'm being dramatic but I'm truly concerned :(