Need advice

Okay so I have been with my new boyfriend just under a month, his ex girlfriend of two years is one of my "Good" friends, it sounds a bit messed up but I asked her if it would bother her and she said no because shes Engadged which was good, me and her are still friends and talk like we normally would. Lately I slept with my boyfriend (I am 20 so no comments on that please I Am an adult) okay going back the more I am with him I think about them to together and it upsets me because I have strong feelings for him but everything reminds me of them! He has a draw full of teddys and presents, he still talks about her quite a bit and the worse thing is he admitted to me he always used to run back to her when she wanted him to I don't know what to do the girl and her fiancé won't stay together I know that because she doesn't even want to be with him but I don't want my boyfriend to leave me for her! But I also feel like I'm in the way of there relationship.... 
* I spent all weekend with him he hasn't spoke about her or anything he keeps telling me how happy he is