My bf doesnt want to get a job..

Our relationship is falling apart and I'm trying my best to be supporting and understand but I don't know when enough is enough. My bf is constantly in an out of jobs and its driving me crazy. He doesn't stay at one job, has lost great opportunities and we don't even live together. I blew up today and called him lazy and we got into a big fight then I told him I hated him (I felt so bad and apologized) then he blamed me and told me all I do is bitch at him causing him to b depressed. He said he NEEDS me to live with him and our baby so he can be motivated, so he has a reason to wake up and so he can come home to us after work. He wants me to live with him and his family again, I don't want to. We lived together while I was pregnant and left when I was 5 months because it didn't work out. My family doesn't like him and leaving would cause so many problems with me and them but he is selfish and doesn't see it that way. Even if I wanted to I know my parents will be hurt and I can't do this to them when they have been the ones helping me. I told him me and my baby girl should be motivating him regardless if we live together or not but he said I don't understand and I really don't. I am scared that my life, our life and future isn't going anywhere. He is stuck living at moms and I don't know what to do or how to push him. I suggested time and space apart but he doesn't want to. He said if I leave him things will just get worse. He promised this time again *sigh* that things will be different and he will go back to work tomorrow, he wants me to call him to really wake him up for work which sounds ridiculous but if I have to do this every morning to push him and motivate him until he gets it together then I'll do it but I don't know if I'm being ridiculous myself. I feel bad leaving, I feel wrong for abandoning him. I know he's suppose to b the man and iv been doing this on my own so far so I don't really need him but every time I try leaving I feel bad and start crying. I want him to help me out and I don't know if I'm suppose to stick around and wait or what I can do..