Idk what to do!!!

my boyfriend and I have been together for almost a year but I still feel so self conscious that he's going to leave me for his ex. probably because when he's mad he'll say he loved her and lost her etc, What do I do?  continue to try and gain it or just leave him?