am i in the wrong?

i started talking to this guy (ill call him E) not too long ago and yesterday i told him me and my (male) friend were gonna go to the mall. at first, E said he didnt want me to go because and we had already decided we weren’t going to talk to other people on the level that were talking. so i wasnt going to go. but my friend is just a friend. then later that day E said that he was cool with it and that he’s just tripping. i asked him multiple times if he was ACTUALLY ok with it, not that he controls my life but i dont want to make him uncomfortable. So he said its ok so i went to the mall with my guy friend. thats it, thats all. now E is telling me he actually didn’t like that and he just wanted to see what i would do. this is making me upset and i just dont know if im in the wrong? i dont feel like im wrong. and it really annoyed and aggravated me.