Do I leave?

I've been with my boyfriend for nearly four years. When we get along its good, when it's bad it's really bad. Last night we ended up in an argument over something petty. But he went on to bring my family into it saying my brother is a druggie (he's only done drugs a few times) my parents didn't want me they threw me out and went on about how much his parents love him and buy him stuff and that I'm jealous. I live with his parents and I'm on my way to getting my own home. He really upset me so I told him that my mother is always telling me I can do better. He went on to say 'I can do a whole lot better I can get with someone who actually have GCSE's and want a good career who's prettier' and all this shit. Now he's begging me not to leave after telling me he wants me gone by the time he gets back from work. He's always acting spoilt and childish when he's with his friends. I go down the local pub with my friends most weekends for a drink or two. But he came down last night and he was running everyone down about how they are all low life's and he was going on as if he's much better than us. I don't know what to do because I do love him and he does a lot for me I just hate him when he goes on like this. Someone please give me advice