Baby daddy control?!

Hi everyone ok so I have been with the father of my child for 4 years we got pregnant and went through abortion then got pregnant again and have a beautiful daughter, but I caught him cheating while pregnant walked in on him having sex with an ex. I left him and didn't speak to him literally 2 wks before my labor. He was gone for about 5 months of my pregnancy. He has been around for the baby ever since and was only wish the girl for about 2 months. Ever since she was born he wants to be a family and move in and be together. I'm just not for it, only him to see his daughter. Advice 😬