I love him

So there is this guy, I fell in love with him and I don't know how to tell him. I've only had sex with him 4 times but we are not together. It's like friends with benefits things sorta. Sometimes I feel like we are in a "love relationship" because of the way we are with each other, well I am with him. He seems to care a lot.

He asked me some months ago if I love him, I denied it because I am not sure if he feels the same way.

Less than a week ago we saw each other. The sex is good, great in fact but I crave more of him. I ask him some questions and long story short he says it's my decision if anything further should happen. I just want him to think things through.

I want to tell him I love him but I am scared. I am scared of rejection.