I want to be close to my son

I have four boys my oldest lives with his father but mostly his grandma I don't get a long with them they talk over me and it's worse with his father he an complete ass so we don't communicate at all my son is my oldest and I miss him so much it's hard to cause when I do call he wants money or to go out he's use to there ways and spoiled and my other gets mad cause the way he acts and doesn't want him around our other boys but I miss my baby he's my first and the more I think of him it makes me so sad I'm not with him and when I call they tell him things and he doesn't talk to me well like he use to I feel like a bad mom to cause I'm so busy with my other three sometimes I forget to call at times or miss his call and what's worse I feel like when he's teenager he's going to hate me