His ex is LITERALLY insane!!! I need advice ASAP!!!

So my bf and I have been seeing each other for a little over 2 months. He’s ex (cheated on him) has recently found out about him and I. She has been following us when we go out and stares at us we were oblivious the first time this happened until one of my boyfriends friends told him about her staring at us at the gas station across from where we work. She has done this on multiple occasions now and I feel like wherever I go or whatever I do I’m being followed by her. I don’t know what to do. Help???