Sex in the middle of the night

I woke up last night to my husband having sex with me. It freaked me out when I asked what he was doing he said he does it all the time. Why would he not wanna wake me up?

Ps. I have to take sleeping pills to sleep at night due to panic attacks throughout the night so I am a very heavy sleeper.