Such a thing as sleep sex?

It's happened about 4 times that I wake up in the middle of the night to my husband touching me then we end up messing around and go back to sleep. well my husband keeps blaming it on me telling me I'm the one who is starting it! I'm almost 100% sure it's him starting it. It's totally random and we are both half asleep and it's at times at like 2am. Has this happened to anyone else?