Not really sure where to add this, but input would be appreciated.

Hi ladies. I'm looking for some input in this current situation. 
We all know our jobs and lives can be extremely stressful at some points. My job requires me to work with one really behavioral client ( severe autism, other mental disorders) and three clients who are no so severe. The lead staff at the AFH that I work at has gotten another job and is leaving. She has worked with this behavioral client for the past seven years. He has attached her maybe 10 times in that span, the most recent being this year which resulted in a broken collar bone. I was not going to take the position nor was it offered to me. They had found someone else to fill the shift and I thought we could train them. However when I learned who would be taking the position I flipped. 
The woman who was going to take it is currently under investigation for a multitude of accusations both from clients, guardians, and staff. I've seen her verbally abuse, neglect, steal items, steal meds, sleep, and just all together not be an advocate for the clients she cares for. I immediately got ahold of my manager and told her if she allowed her to step foot in this house that I would be done. I will not stand for any type of abuse. These people already have difficult lives and it's important to keep them thriving for better things to come. 
Ultimately after going round and round with my manager and telling her what I thought, it either came down to me taking the position or letting this other woman come in. I chose to take the position. The thought of anyone being mistreated at my place of work and not having any repercussions for her behavior absolutely appalled me. I know I stood up for the right reasons. I know I did the right thing in terms of these clients lives. However, I'm not sure it's the best for me. 
My husband and I are currently ttc. I had a chemical pregnancy last month and while it was difficult to get through, my husband and I came out stronger. The thing is, is that it is naive to think that this particular 1:1 will never attack. During the day there is no one else with you. You deal primarily with him alone. What really scares me is the fact that we are trying. We will get pregnant at some point. I'm sitting here thinking and questioning if I did the right thing. At some point, no job is worth your health. At some point there has to be a line drawn in the sand. 
I do not want to let him down, or the other clients down by allowing a toxic employee into the house. My manager said she simply needed to fill the shift. However I can't help but wonder if I am setting myself up for failure. To not put myself first during pregnancy would be stupid of me. If you add in the fact that I would be a high risk pregnancy and I feel like all my efforts would be null and void. I am simply just second guessing myself. Any input would be greatly appreciated. Thank you.