He’s only human, so it’s alright to cheat?

I confronted him about his past mistakes. More than once has he done me wrong he doesn’t want to be in a relationship but wants to continue to see me & have sex with me? He responded with he’s only human and isn’t perfect..

He also goes to church with his Christian sister and told me we cannot continue to have sex anymore because his sister said it’s wrong.. but it’s alright to go out to clubs and drink?

I’m confused I feel like I’m the problem and something really is wrong with me like he says

I would also like to add that we have a 4 year old together and have been together for almost 6 years now