My lawyer has told me to hold my daughter

Please no hate comments please hear me out before judging me. Idk if this is right group to post in but my god I need to vent. I feel horrible. These past 8 months since my BD and I broke up have been hell. It’s consisted of court, CAS, doctors, severe diaper rashes, full body skin reactions to food she’s allergic to, extremely bad heat rashes with eczema flare ups all while in his care for a short period of time. (I have documented everything) he’s never cared much for his daughter after she was born and after we broke up it’s gotten so much worse. He lives with his mother in a one bedroom apartment. He refuses to pay child support for his daughter and he decided this not me but he only has visitation Tuesdays and Thursdays 4-8pm and every other Sunday 8-8pm. This was at his request. We were suppose to have court this past April but because if Covid it didn’t happen. I just finished school for PSW and got a job at a nursing home to support my daughter. He harassed me for told me it’s to high risk for our daughter so I took a leave of absence. My lawyer wanted in writing a list of precautions both parties are taking to protect the child. I told him everything about the boyfriend staying here to avoid exposure risk to where he’s working and what we are doing and she doesn’t go shopping. He wrote in the book simply that he showers after work that his mother doesn’t have a job and neither does his gf. He didn’t take it serious and then two days later dropped that his girlfriend is now working in a nursing home. Gave the name but no other information at all. Found out through a mutual friend she’s actually a Plan A employee so potentially working for a few different nursing homes. so he also lied to top it off and his work was positive for covid but he never notified me after having our daughter so my lawyer has told me to stop all visitation at this time in the best interest of my daughter. My lawyer finds my ex to be all over the map and not taking this covid very serious at all so he firmly believes it’s what’s best for her. I’m struggling to cope with this. I feel horrible when I know I have no reason to. I understand a child needs both parents in her life but he hasn’t been much like a father at all since the day she was born and got worse when we broke up. I do feel guilty because I would be devastated myself if he took her from me but I know I’m doing what I can to make sure she is safe. Idk if I’m looking for advice or just needing to vent. This is so hard I just want to protect my daughter and do what’s best for her and honestly at this point I don’t know what is going to hurt her and not.