What Should I Do?

We had a bit of a past. We got in trouble. He had to move and I had to take the fall. I was the laughing stock and my reputation was ruined, but the whole thing was my idea. But that was 4 years ago. After maybe a year or two of me ignoring him we finally became friends. We both grew up a lot, especially me. Now, he tells me that even after all these years no one has ever made him feel the way I did and that he's still in love with me. I think...I know I feel the same way..but I'm afraid of what people would say if we started dating again. What should I do?