mixed signals

So I been talking to this guy a couple weeks now. I've known him for a year. And he seems so interested in me at times, asks me about my day, how I'm feeling, what my favorite things are... etc. but he says he isn't sure about a relationship because we're so different.... We agreed upon a sexual relationship.... mainly for me because I really like the guy. And sometimes when we're just talking I feel like he likes me to... but I wonder if he's afraid of me. Supposedly he was an ugly duckling, and he was just astounded when I told him I was into him. He literally could not believe it. I gave him my number while he was at work. And he was like stand there Ima make sure you didn't give me a bogus number. And he actually was just speechless that I wasn't joking. So I feel like I'm getting mixed signals.... help