HELP!!!!

i had sex with this guy more than once, strictly that, no intentions of dating. He has a younger brother who is so sweet and the picture perfect boyfriend he would always flirt with me, and he found out about me and his brother but is STILL flirty with me, we have been hanging out A LOT like it’s literally been a week straight, i’m regretting doing anything with his brother so bad because now i messed things up, what do i do???? FYI our whole friend group would tear apart if i didn’t anything but that’s all i’m thinking about