How to confront him in a mature/non hostile way?

So I have a friend who always flirts with me whenever I see him. He always says how he wants to have sex with me and I always turn him down. He says im the only girl he's interested in even though in my gut I know that's a lie. I rarely ever talk to him when I'm not visiting his place (His place is like a bachelors pad and a whole bunch of people come over to hang out) I saw a girl message him about someone missing him and how he misses them too. I don't want to straight up call him a liar. But he's always saying how he "Really doesn't just want sex." I don't want to be mean but at the same time isn't he being mean to me by lying to me? Help? How do I approach this in a mature but firm way?