Am I being selfish ?

I was with my ex for 3 years on and off. I thought he was the love of my life that I was going to marry him and have kids with him. But he was an irresponsible, uncaring, liar, a cheater and everything that you can possible think of. I haven't been with him since April of last year we try to fix things a couple of times this year but he just didn't care about us. Anyways I went out with my friend and her boyfriend about a month ago. She invited one of her close guy friend which happens to be my exs friend. I danced with him and we had a good time he kept flirting with me and asked me about my ex I told him what happened and he started saying how stupid my ex was for doing all that to me. I've alway find him cute but I never thought of him if anything than just a friend. Well as we were dancing he kept grabbing my hand and kissing on my neck. So the point is that we had sex that night and I dont know how it happened we both started to have feeling for each other he tells me he likes me a lot and he wishes things were different for us. But we can't be together because of the people that are involve I know is wrong but am I being selfish for putting my feelings first than any one else ? I know that if my ex finds out he'll go crazy but I like this guy and it's so wrong but the more that I try to get away from him the more I miss him and the harder it is. I told him to just block me but he texts me once in a while telling me he miss me and that if he could fix things he would but my ex is still his friend and he just feels bad about what we did. *(he's the second guy ive been with) maybe that's why I feel so bad ? What do you guys think ? I need some help ladies 😞