Honest race question (looking to learn)

So I was just meeting some new neighbors and we were all joking around. One of them told my roommate to eat a chicken wing and I was like no she can’t have that she’ll throw up (she’s been vegetarian for two years and the one time she ate meat she barfed for hours) and he didn’t believe me. So I said “boyyyy believe me” and didn’t get to finish. He said by calling him boy I was calling him slave. I was taken aback because I was actually really digging him and thought we were cool. I playfully call all of my friends boy or girl (depending on which they go by) and have never heard of such a thing. Now ignorance is not bliss and I want to know if I’m wrong here but everyone I know no matter what the color has been called like this by a parent. None of my black friends have ever told me this is derogatory. If I’m wrong I’ll correct myself because no one should be made to feel less than but I’ve honestly never heard someone be upset by this before and I don’t know if this is a him thing or if this really does hurt people. Maybe I am just ignorant but I’m honestly at a loss. We were getting along so well and I’ve never heard anything like this from being so playful with someone. I’m so upset because I’m the farthest thing from racist and to be called as such hurts when I fight against it everyday. But I can learn from my mistakes if I’m actually mistaken. Accepting all criticism or anything else anyone has to say.