Should me and my boyfriend breakup????

Ok so we’ve been together for almost 6 months and yes we’ve had sex n stuff but here for the past few weeks he’s been acting kinda different and like now every time I’m around him my heart hurts. I tried to break up w him yesterday and he started crying and I felt bad bc I was making my baby cry and so I stayed. And he don’t cry. He’s my first real boyfriend to where we actually went places n stuff and I lost my v-card to him. My mom is stressing me out ab him and idk if that’s why I wanna leave. I really need help ASAP. I don’t know what to do.