Walking away

Lately I've been talking to this guy and he's a couple years older than me. Everything seems perfect and it's going along smoothly. We haven't had sex although we have talked about it daily for the past two months. I am falling super hard for him and I know that the end result won't be what I really want. So how do I walk away from this guy, who seems to be everything I've ever been asking for? It just seems wrong to me to not even try and pursue it anymore. I mean who knows, it might end up working very well. But I have a feeling I will be hurt in the end. Btw I work with this guy and he's my boss. He's stated multiple times that I won't lose my job regardless of what happens to us and he'll always be my friend bc we were friends before this. So I'm not worried about that. He alway tells me that if I ever want to stop this, or break up I suppose, he won't judge me or act differently towards me. So what do I do? I don't want to hurt his feelings or make him feel like it's his fault.