The most hurtful thing is seeing my baby cry for his daddy

So I met my sons father in high school we were both the same age (16) this was in 2009 we stay with each other on and off for 7 years by 2014 I had gotten pregnant and I had my first child my son was born in October thru out my whole pregnancy I had to go thru it alone it was stressful when I told him that I was pregnant he immediately told me to lose it get rid of it because he wasn’t ready for a baby so I said I can’t get of a baby I believe that you should never have an abortion so I lived in a house all by myself did everything that I needed to do for me to take care of my son never did he come in and say hey do you need help or anything everytime he saw me at the store he would turn around and leave so my son is now 4 years old he does go to see his grandparents which is the parents of his father my son knows who his father is so yesterday I let my son go to his grandmas house so she calls and tells me hey I’m going to fathers house is it ok that I can my grandchild I told her yes you can take him but when she was going to leave my sons dad shows up and says hey let me have him I want to spend time with him so they left not even an hour I get a phone call saying hey I need you to come pick your son I’m going out I had told him spend time with your son going out can wait he said no I need to go out with my friends when he does this every day so I said ok I’ll get him I was so mad because he would rather be out drinking than be with his son so I go get my son and he sees me and he goes running to his dad so he picks him up and he hugs him but my son is crying so hard that it makes me cry and all my son is saying my daddy my daddy I want to stay with my daddy and I look at his dad and he tells me to leave with my son he just laughs at me because he sees my son crying so I put my son in the car seat and we leave and my son is crying so much and he’s there telling me to go back to his dad he wants to be with him oh how much it hurts me to see my baby crying he did not stop crying his dad hasn’t really been there for him and my son has gotten so attached to him that he didn’t want to let him go how do I get a guy to realize that his son is crying for him and wants to be with him and for him to stop doing what he wants to do and be a dad for once he doesnt realize that he is hurting my son and it hurts me to see my baby hurt and crying for his dad he cried the whole night for him I had tried to play with him to forget about what just happened but no matter what I did he would just cry I just want to know what to tell my son about this situation what should I do should I just not let my son go anymore to not see him hurt anymore or should I just let him go