What would you do? Absent biological father question.

I'm really trying to think this through before I make any decisions and I could really use some advice. Anyone that I talk to in person about this is pretty bias on my side so I'd like your opinions. I had my daughter at 16 with my ex that I was dating for about a year at the time. I was very naive as a teen to say the least. He was 18 at the time and not much smarter either. He cheated on me MANY times throughout our relationship and while I was pregnant but I foolishly stayed with him. Long story short, I found out he was very into pills after some time (opioids) and at first I was totally against it and would never try it with him. After a surgery I had when my daughter was 6 months old I was prescribed Vicodin and after taking them I didn't think they were so bad and I began using with my ex. My father died suddenly when I was 17 and I basically gave up on life at this point. This turned into a 2 year struggle with drug addiction leading to heroin use with my ex. My mother picked up the slack with my daughter. My ex moved away to "get clean" when she was 2. He ended up using out there and stayed for 4 years. When he left it took a while, but I got my life together. I have been sober for 5 years now and I got married in 2016. Life has been great! My problem is, my ex came back to my state about a year and a half ago. He wasn't clean but after going to jail he took me to court for custody claiming to be clean. Eventually he was granted supervised visitation with mandatory drug testing. After seeing her 2 times he failed a drug test and lost all visitation rights. I was sad for her and him, honestly. It's a sad situation all around. So, I recently got a message on facebook from his sister saying that her and her father totally understand that I don't want my ex around her but they really want to see her. I was really considering calling her to talk. The problem with his family is that we had a major blow up when my ex first came back and his dad showed up to my moms cussing her out because my ex hadn't been able to reach me to see our daughter. It was crazy and uncalled for. He hadn't been around in 4 years! Anyways, I was just getting ready to give it a chance when I get another message from my ex's apparent new girlfriend saying that they would love me, my husband and our daughter to come to her sons birthday party May 5th. I don't even know this woman. Now, I don't trust his dad or sister. Last I heard of my ex he failed a drug test and was back on drugs last year. I feel bad in a way for his family but then, I don't. WHAT WOULD YOU DO?! Should I give his father and sister a chance for my daughters sake?

EDIT: I forgot to add that the main reason I worry about his family is because they lied to me and tried to hide him coming into town when he did because they knew I did not trust him nor want my daughter around him until I knew he was off of drugs for sure. The sister is only 18/19 now. Also, his supposed girlfriend was also a known drug user in the city that we were from. I've been there so I'm trying to not judge or assume that she is still into those things but that also makes me weary.