Trust issues?

Given some previous personal experiences, I have a difficult time trusting people in general, regardless of how trustworthy they can be. My SO tries to remind me ever so often that I am the one girl for him, yet in the pit of my stomach, I can't stop being worried when his ex tries to contact him, when girls try to flirt/offer him sex at parties, when he shares housing with female friends, and when he goes out with his single guy friends who want to pick up chicks. Even in those situations, he invites me over to his place often, as well as invites me to go out with him and his friends. This is my first major relationship -- how does one get over underlying feelings like these and just trust for once?