Am I in the wrong here ?

So about two weeks ago I got asked to homecoming , I said yes but I didn’t really want it go with the guy, he’s really nice and stuff but i would rather just go with friends . Before he even asked me with a sign and stuff he asked if he asked if I would say yes , I told him that I wouldn’t and he made me feel really bad and I didn’t want to hurt his feelings. So fast forward to after he asked me , I talked to him AGAIN telling him I really didn’t want to of with him and he got sad again and made me feel bad so I changed my mind again. Well today I was thinking and I honestly just want to go with friends , like it would be more fun for him to go with his friends anyways because my group was all girls and he would of been the only boy . So I told him I didn’t want to go and he got sad and said okay , I didn’t give in this time and told him I’m really sorry and just think it would be more fun if we didn’t go together . Now his family is really mad at me and I feel terrible . All of my friends told me I should do what’s best for me and that’s what I did , do you think what I did was wrong ? :(