Leave or stay?

I have been talking to the same guy for about two years. We dated and the relationship was truly a mess. He tried talking to other girls, drank a lot, never helped me with anything, and I finally called it quits after he drunkly told me I was worthless in front of all of his friends. We started talked again just about a year later. He insisted he changed and now gave up most social medias. Has been there for me through everything. He supports me financially as needed. My family found out I’m talking to him and they won’t speak to me and now it’s gotten to my friends too. He is saying that if they don’t want me to be happy I shouldn’t talk to them. My friends and family all say he makes me a different person and that he is trying to isolate me. He doesn’t like when I spend time with friends. I also found out he had an only fans which kind of upset me. I don’t know what to do I don’t know how to get away from him even if I wanted to. Is this toxic I don’t know. I have grown up around toxic my whole life. He also needs my location or he freaks out and he also has broke and entered before.