I am having an affair

So I am that asshole yes I know. I am posting this knowing I will recieve hateful comments. I started dating this guy years ago and it didn't work out due to his commitment issues. I ended up moving on and met my now husband. I was very happy with my husband for a long time. He is a great guy but doesn't see the importance in showing me attention or affection. We act like friends 90% of the time. We will go weeks and weeks without sex and go weeks without really kissing. I will get a peck sometimes but that's it. He kisses his daughter on the lips more in one day than he kisses me in a week. He never texts me during the day and when I get home from work he just sits there and watches tv. I have talked to him about this several times and it always turns into a fight and it never gets solved. The guy I dated before began texting me again a year ago and at first it was just friendly. We would just say hey how are you every now and then and that's it. Slowly it turned into flirting on both sides and after 6 months of me telling him no to meeting up I finally caved and did it. We started hanging out once or twice a week just casually. Eventually we started messing around and just recently we started having sex. He knows about my husband obviously and he wants me to leave him. I got myself into this and I am trying to get out. I do have feelings for the guy but I think I need to confess to my husband and either try and work it out or let him leave me. My dad cheated on my mom and my entire life I grew up hating him for it. I have been cheated on by exbfs and know the pain it causes. I feel like a complete bitch and asshole. I never thought I would be that person.