Toddler asking where his absent family is...help😞

My family has never been very supportive and when I thought they had a change of heart I moved 8 hrs to be closer to them. It was a mistake, they are toxic as ever. They barely participate in my life. My son is almost four and has started soccer. Recently he’s been asking where they are and I tell him they are busy or they couldn’t make it but he seems to be realizing they don’t come around much. Do I just keep telling him they’re busy every day? I feel like he’s getting tired of this answer and I’m trying not to bad talk them but he’s asking more in depth questions and I’m running out of basic answers😔 has anyone had to deal with this? What did you do?