Ending it & being happy

I met this guy when I was super young and he was the first guy I ever really got serious with. He was more experienced than me and I found that super attractive. I lost my virginity to him and him and porn basically taught me everything I know about sex. We were extremely comfortable with each other. We dated for a year and a half and had a really bad break up. It's 3 years later and we still have casual make out sessions and he always tells me how he wants to be serious again. I like him but he's also very ugly and very fat and very annoying to everyone I know. I love his personality but sometimes I get the weirdest feeling around him. He thinks we are talking but I don't know how to break it to him without making it awkward between us.