Taking a break. What does it mean?

Ok so I've been seeing this guy for 4 months now and things have been going great. A few days after meeting him I find out that he is cousins with 2 of my friends. So we both agreed that that isn't an issue however he would never want to hang out with them because they're younger than him. Months pass and things are going great until the cousins realise that we're together then suddenly they have a problem with it and now my guys telling me we should take a break because his cousins are mad thats we're together. Am I wrong for thinking that its silly of him for listening to kids that are younger than him or should I just not enter his family affairs? From the situation read here do you think he was being serious with me or was he just having fun for the sake of having fun until he got a scapegoat?