Relationship help

My boyfriend and I have been together for a couple years now. We’ve worked through every issue we have had but this time it has to do with family and I have no idea how to get past this. I have a 4 year old nephew and we bond so great. He comes over every other week late Thursday-Sunday. I’ve never been around kids my whole life so my sibling having a child was super special to all of us. He’s the only little one in our small family. With that being said I give him so much love and attention and really take care of him and try to play with him when he’s around. He has no other kids to hang around with in our family so I just want to make sure he’s entertained. And either way I love kids but I’m never around them so I try to enjoy every moment I have with him.

My boyfriend on the other hand has been around kids his whole life. He has a huge family and he currently lives with 3 cousins from 6y/o to 13. So he’s always around them. I have always embraced his little cousins throughout the years and love spending time with them as well. We go out to do something sometimes and he asks them if they want to go with us or I’ll ask them and he will never complain.

When it comes to my nephew he’s the complete opposite. He never wants him around. He gets mad when I ask him if we can take him with us to run an errand. He hates when my nephew is on top of me. Just recently we went on my family trip and he had been mad the entire time because my nephew always wants to be near me or sit next to me. Or I’m always giving him attention. He says he doesn’t like that and he doesn’t want to babysit. I have expressed to him how I don’t do that to him and how he’s very special to me because he’s the only child in my family.

He is a hyper child but I think that’s how most kids are and he’s attached to me because I show him the most care and entertainment. He’s 4 but he can articulate himself super well and knows how to behave. Again he just has a lot of energy.

We can’t seem to get past this issue. Whenever it comes up where I want to take my nephew with us or he’s around, he gets mad and we get into an argument and he says I don’t listen. I don’t know what else to do. We have always gotten past any issue we’ve had but this one involves family and I feel it’s a dealbreaker. He’s not being understanding and says I don’t listen but family is so important to me, especially if we’re to have a future together. I don’t want to ignore my nephew and stop caring for him just because he doesn’t like it.

Please help