am I dumb for coming back for the sex?

So I recently hooked up w my coworker (never thought I'd say that) but I was very drunk the first time. I'm not gonna lie, I was very hurt the first time we had sex bc I felt like I was too drunk to consent. That night I cried and pushed him off and drove home.  I wanted sex and I also didn't want to have our first and last time together to end the way it did the first time. I wanted to actually enjoy it so we both agreed that we would finish what was started the next day. We met up the next day and it was honestly the best sex of my life. I actually reached climax which I hadn't done w previous partners. He was doing things like kissing my body and he was just super affectionate in general!..well that happened and now I want sex from him again. Idk how to ask him or if me going back to him will make me become attached..I just don't want to increase my body count. Advice please!!