Boyfriend and ex girlfriend

So my boyfriend used to date one of my old friends. It wasnt a proper relationship, but she was his first kiss. It kills me that she was his first kiss, when I had the chance but turned it down. The worst part is though they were, before and after dating (including breaking up) really close. I hate that their still so close and I know for a fact that she still likes him, as she told another friend and that friend told me.

I don't trust her anymore, and I'm afraid she might try and make a move on him when theyre spending time together.

Worst part is, she set us up knowing we liked one another. She tells me how cute we are together, yet I know she reallly likes him. Am I just being stupid?

Oh and by the way, my boyfriend knows that she likes him, and I think he likes it.