Time to breakup?

How do you know when it's time to break up? I know this is a silly question. I always thought that I would be the girl who would leave when she was unhappy. But, I just feel so dependent & I love him deeply. We just have nothing in common- he is my complete opposite. I feel like he brings me down :( He has social anxiety and is a very harsh person. We've been together 4 years, are in college, and we live together so it is hard decision. I don't know what I would do without him but at the same time I feel like I could accomplish so much and meet so many new people, & have new adventures. What do I do?