What to do?

Ok so I've been sleeping with this man for 3 months. The sex is usually reeeeeaaaalllllllyyyyy good. I mean like best of my life. We both feel this way. We aren't in a relationship & never discussed what we want from each other until the other day when I found out he was still in contact with his ex. Even though we never spoke about what our situation is we agreed that we were sexually exclusive but he told me that when she comes to town (she lives out of country) they always have sex. Now even though I don't want to be in a relationship with him, the thought of him with another woman bothers me. I decided to keep dealing with him anyway cause like I said I really don't want to be with him anyway, it's just sex. I've seen him twice since that discussion. The first time I could tell something was off about the sex. & I wasn't even thinking about the other woman or the situation so idk what the issue was but I usually get wet really fast & it took a few minutes this time. But once we got over that bump, the sex went fine. Something was still off though. Then the second time he just couldn't get it up. Again, nobody brought anything up about the other women, it wasn't on my mind & I don't think it was on his. To be honest I feel like it's my fault. Like maybe I did something to turn him off. Or maybe it's cause we've been having sex like every other day, some days back to back for about 3 months & it's just not exciting anymore. Or maybe us having to discuss somebody from his past changed our dynamic but either way I need answers. What do y'all think the issue was? Or maybe it just happened & it's nobody's fault? Has this ever happened to y'all? What do I do to fix it cause I wanna go back to how the sex was before.

Edit: I do like him but he's not good for me. I don't want to be with him because I would only get myself hurt. If he was more responsible & had his life together a little more I would consider it but no.