moral issue I think 🤔

me and my ex have a 9yr old together he and I haven't been on talking terms every since he put me and my kids out his house so he could continue to be with the girl he cheated with and had a new baby by . now his other 9yr old ( crazy right) by a totally different woman is having a birthday party this weekend at his dads house and wants my son to come. now we live 4 minutes from my exes house and he hasn't seen or called my son in 4 months . my son doesn't want to go but I think he is just saying that cause he doesn't want to hurt me. do I let my son go ?