Consent issues/ rape

So last night my partner tried to have sex and i had just worked short change shifts and was so tired so we just went to sleep and i woke at 4am and he was rubbing my clit and vagina and trying to get into my underwear and so I moved and he quickly pulled his hand away until he was sure I was asleep and did it 3 more times making sure I was asleep every time. Is this normal and am I over reacting thinking its not ok when I said no? Thanks 💕