Abusive bf

We have been together for 4 years, hes the only bf I have ever had and I love him. We have recently been trying to conceive. He is most of the time really sweet we hardly ever argue, but when we do he becomes very mean at times, like calling me names, yelling at me, or puts his hands on me. He doesn’t necessarily hit me but sometimes he grabs me ugly or pushes me away kind of rough. Idk if this is really abuse or not but would you leave your man if he acts in that way?