End it before it even starts?

So I'm seeing a guy right now and he is incredibly sweet. Total boyfriend material. Only problem is I'm not actually attracted to him. He's just not my type. But on a regular basis I see another guy who I am attracted to and think that if he were to approach me I'd go for him instead. With all that said should I just cut it with the guy I'm seeing now? He's great, so I'd feel bad for just stringing him along. Or am I being vain and should I just make the best of an already-good thing??