Father/daughter relationship help

Sarah

My husband has never really tried to take care of our daughter. When she was a baby he was always quick to hand her back to me rather than try to calm her himself. He had never been around a baby before so I always quickly gave in. When he is home from work I always still do everything for her and he zones out into his phone or tv.

Now that she is 2 she never wants him around (will tell him to go in the other room and even push/hit him). He is reluctant to watch her because she is so against him but I think the way for them to become friends is for him to watch her more with me home and with me gone. He will say things like him being there more won't help so he doesn't even try.

Any thoughts on how to strengthen their relationship? Am I correct in thinking her being so against him being around is because the lack of him being around?