Signing over his rights..

So I'm married to my daughter's father since 12/4/09 but we haven't been together since 12/17/11. At which point I caught him in bed with his new baby mom.. Our daughter was born on 6/29/10 and he was a great father to her.. He was always holding her and doing the late night feeding.. It was great to see how much she needn't to him and how much he loved her.. She was a daddy's girl too!! Well now his filled for our divorce which is great.. He hasn't really been in my daughter's life since he got with his new baby mom to the point my daughter has seen her brother like maybe five times in two years.. By his choice. So I called them and was like all I want is full custody of my daughter at which point her father says I'm signing my rights over because that is your child only I have never loved her and she means not a damn thing to me.. At which point I started crying because my daughter wants to see him badly.. He than goes on to say don't she call your boyfriend daddy (I've been with my boyfriend two year but he been my best friend for 12 years) so I said yes he said than that's her father now leave us the fuck a lone.. Does it make since I'm heart broken over this?? I mean he hasn't been around in forever but still that's her bio father.. Oh and he wants her last name changed asap..