I think he'll leave me... Again

Hey ladies, so basically my ex boyfriend came back into my life around 4 months ago. At first he was full on trying to impress me and win me back, huge romantic gestures, telling me how he feels ect. But now I have let myself fall for him again he seems to be backing off, like majorly. We're not 'boyfriend' 'girlfriend' or anything but we talked everyday and have a phonecall every single night.. As much as I think I'm being paranoid about the whole situation I can't help thinking he's going to leave me again (although he swore he wouldn't do it to me twice) I'm terrified because after our last break up I couldn't quite get over it, we spent 2 years apart, all of which I stayed single and couldn't bear to say I love you to another person and I thought about him every day. Honestly, I'm not entirely sure I'd be able to get over him if he was going to leave again.