Boyfriend has a creeper

Samantha • 92F in the U.S. Army
Okay so I lived in a different state than my BF before I met him. So he has gone to the same school all 12 years. I have moved multiple times and by luck ended up at his school. So sophomore year I was new there and met him. Instantly had a crush on him but he was in a complicated relationship with this one girl few months later they break up and I confront him with my crush a few days after their split. Obviously he said yes since now its 2 and 1/2 years but when we started out his exes friend would bother him about moving on so quick and recently she has been stalking me at school and bothering him in classes they share. His ex moved on so I don't know why she is still hating on us. Does she like him? Has she had a crush on him the whole time and just been waiting for her friend to move on? She is definitely stepping into my territory because he was sitting with the school nurse one day and as he told it he couldn't wait to get out because she kept staring at him and making comments. We are very happy how should I deal with the creep?