Visitors after birth

My husband and I both have large, loud families and we want to limit visiting after our baby arrives. I know some people in the family will get upset or offended by this. For other people who are limiting visitors, how did you go about it??