Wrong for being with her dad?

So I just turned 16 and I have a 2 year old daughter. In January her dad/my boyfriend broke up with me because he needed space from being a dad and stuff like that. I know that it was wrong and that he shouldn’t have acted that way so I did things like not inviting him to her 2nd birthday party because I didn’t think he deserved to come but now he wanted to step up again and help out again and be involved again.. I never fell out of love for him or anything so when he asked to come back into our lives I said yes and it sounded like he was serious. Now that we’re dating again my family and friends that I told are shaming me for getting back with him and telling me he’s a bad person and that he’s gonna hurt me and break mine and my daughters heart again. People don’t get how hard it was for me to hear her call “dadda dadda” and not get him to hold her or play with her. Am I a bad person or doing something wrong?