How would you end a casual relationship?

I have been intimate with this guy maybe 4 times in the past year. We've gone on a handful of dates-- maybe 7. We've tried to see each other a few times but neither one of us is willing to inconvenience ourselves. We went out recently and we were intimate. But he's still so guarded and rigged. I will get emotionally attached if things continue this way. I want to end things but what's the best way? Please explain in comments.