Should I be mad or?

Alright lovely ladies I need some advice,

My partner of nearly 4 years, his ex gf who caused drama for us when we first got together, followed him on instagram I told him to accept it to see if she had something to say.

Well then she's messaged him about the dog they brought when they were together and that soon turned into a conversation, she's added him because her man fucked up on her.

now when I've asked him to remove her his refusing to do so, because "I shouldn't have told him to do accept her so it's my fault" and I shouldn't be angry about it. I should trust him. But I seriously can not get it out of my head about I'm literally about to call it quits with him. I can't deal anymore.