Boyfriend annoys me on purpose and won't stop?

My boyfriend has ADHD and will do things to make me mad. He will touch me inappropriately and I'll tell him to get off and he won't. He will scream all day. He acts like a child. Am I wrong for wanting to leave?