Please help!

So my ex and I had a falling out back in January while I was pregnant. Then in March we made up and talked through everything, but my family still hates him, they think I shouldn’t even let him see her everyday. Now having our daughter and us trying to get back together, my family still disrespects him and I hate it. He said I can fix it by not allowing them to see her, but I still live with my family. He suffers from depression and anxiety, and all of this has stressed him out completely. This is making him have suicidal thoughts. I don’t know what to do y’all, I’m completely drained from all of this. I love my family and I love him. I want him to be happy but I’m too nice to keep my daughter from my family. I’m so conflicted.