Save our reletionship

My fiance and I at the beginning of our reletionship was a rocky staty, jealousy and immaturity got in the way but three months ago when I moved it it was going good intill he would throw the past in my face and make me feel terrible. Mind you he has done stuff to me I let go and forgave. Well, yesterday he told me he wasnt happy in this reletionship but he wanted to make it work and we just needed a fresh start, so I agreed. But I'm so worried that we cant fix this, or one day I'll wake up and he will just tell me to get out, keep in mind I have no where to go. Whenever I ask for reassurance he just gets annoyed and says "Are you seriously trying to fight right now?" When I'm not...how can we make this reletionship work?