Being introduced into a new culture

I'm 18 and so is the baby's father. I am black Vietnamese and white. I've been raised a pretty Normal life. I've never had any out of the ordinary tradition. My baby's father is Nigerian. His family is huge which is something I am Not used to. In his culture they do so many things differently. The way we were raised is a big one. His family wants to take the baby after I give birth in October so I can focus on college. I plan on staying in school for 6 years (getting my masters) I'm afraid that with the baby being raised in a totally different environment that when I finally get him back the adjustment may be hard. I'm not for sure if I'm making any sense, but has anyone else gone through this?