Don't want partner near baby

Would anyone else feel the same way....
I fell pregnant and my partner was not happy about it, I waited until I was 3 months gone before telling him as I knew I could bring the baby up as a single mum. I gave him the news and he looked as though he was gon g to cry, his first words were "well it's not to late to do something about it". I told him there was no way I would do that and I was really excited to be pregnant. I also said if you do not want to be a part of it then we can spilt up and I can manage financially on my own. He was no help throughout my pregnancy, didn't come to any scans or hospital appointments and never asked how me or baby we're doing even watched me shifting heavy boxes while I was going through my nesting stage and told me to "get on with it" because he didn't want to help. But now the baby is born he wants to call him "my son and my boy" all this is driving me crazy and it feels so false after his behaviour throughout . I barley let him hold him or do anything for him as I tell him, if it wasn't for me the baby wouldn't be here today because he didn't want him. Should I get over this or would anyone else struggle as I am xx