Sleep sex??

So Idk if I should be concerned or not but my husband has sleep sex with me!

So one night (this has Happend on multiple occasions) we had sex and after we went to sleep. So about three hours in my sleep I wake up to him on top of me. Of coarse I allow it who doesn't want another round. But the sex is a little more aggressive and when he's done he goes right to sleep. So I tell him about it the next day and he looks at me like I have 3 heads. I mean he doesn't remember any of it besides the first time. Does anyone else have this problem? It's happened more than once, but I think only when we sleep naked. Can anyone give advice on what I could do to keep this from happening often?