Honest thoughts? Venting

My boyfriend and i have been together for awhile now. We have grew up together but hung out on and off over the years. We were like family till we started dating. Ive had my bad moments due to alcoholism with no cheating involved. The first few months of our relationship was rough i admit i had a big part in it. Through all of that he still wants to be with me on top of that we are trying to TTC. Only thing is his family holds those things ive done to him against me. I know i cant make up for the hurt ive caused him but we are still together. Ive been sober for a few months now and he still drinks on and off but not as bad as he use too. He is a family man and loves his family but since we have been together he stopped communicating with them. His sister just got out of prison and is threatening me because of the pain i caused him. I don’t talk to any of his family or friends because they think I’m the worst person. Their trying to break me and him up and we ignore it. Lately I’ve been going through some things and been super stressed out. Should i leave him because his family dont like? Would that be reasonable? Or we should stop TTC because of it? I really need some honest opinions.