Why are people against immigrants?

I'm just wondering why people are against illegal and sometimes legal immigrants. It doesn't make any sense because the USA was built by immigrants. The only people in the world who can have an actual problem with it is people who are native to Eastern Africa because that's where humans started.  And I get that yes they're illegal but most of them are here for their family and children.