Boy asked me out

So I’m 14, and a boy in my grade asked me out. He’s really sweet and I said yes, but I don’t particularly find him physically attractive. My friends are all saying I could do better, but he’s really cool. I really would like to hang out with him and get to know him, but I’m not sure how much further I would want it to go. Am I leading him on? Should I have said no?