Should I stop talking to him?

Okay so there is this guy and he is very sweet almost like the perfect guy but he really wants a relationship and I don't feel the same way. Yet, all of my friends are telling me to continue talking to him because he's a great guy. I just don't have the same feelings he does. My ex boyfriend who is also a very close friend that i kinda always had feelings for says i should drop him. Im just not sure what to do...