Conflicted feelings

My boyfriend and I broke up a few months ago. We were together 3 and a half years and I was crazy we both were about eachother. Anyway he was really depressed and down after a month apart and began speaking to me again. He's been trying ps hard. He's brought me out and dates and bought me such a beautiful Christmas present that cost him all his money (I did t ask him to it just makes me sad he did.) Problem is I go up an down about how I feel. I would never dare say it to him as I know he loves me so much and it would destroy him. I don't know am I making these doubts up or I should act on them. I don't know if we gave each other the chance to get over one another. I don't know if staying together is right or wrong or we'll regret it in years to come. Today a guy asked me out and I had to refuse because I know if ex Boyf found out he'd be devestated. Has anyone felt like this before? I feel geuninely down when the doubt comes in to my head like I'm living a big lie to him. I feel horrible.